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Introduction 

Volume 12: Experiences with 
Standards-Based Reform 



by 

Alison Simmons 



Editor 

SABES Central 
Resource Center/ 
World Education 



I n this volume of Adventures in Assess- 
ment, teachers and practitioners write 
about their experiences with standards- 
based reform initiatives at both the state 
and national levels. In the process of trans- 
lating these initiatives into everyday prac- 
tice, they have come to a better under- 
standing of their students, curricula and 
assessment, and the importance of getting 
students and teachers involved in identify- 
ing what it is they need/ want to know and 
for what purpose and context. 

Sherry Spaulding and Roseann Ritter 
were part of the implementation phase of 
the Massachusetts ESOL frameworks 
project. They write about their involve- 
ment in the project and how it helped them 
better understand the role of standards in 
developing curriculum and the importance 
of developing ways to understand what 
students know, and want and need to 
know. 

Beth Brockman participated in Phase 
Two of her state's framework initiative in 
North Carolina. In her look at the draft 
frameworks developed by practitioners in 
her state, she focuses on the non-language 
outcomes of her students and how these 
could fit into a framework for assessment. 
For more information about the North 
Carolina ESOL Frameworks contact Lit- 
eracy South, (919) 682-8108. 

At the national level, Peggy McGuire, 
assessment coordinator for Equipped for 
the Future (EFF), leads us through EFF's 
assessment framework. She discusses the 
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development and the implementation of 
the framework and future activities. She 
helps us see how it is linked closely to 
standards and how assessment is more 
than just one dimension, one standard and 
one tool. Their approach can help us see 
how this initiative can connect to our own 
state initiatives as well as our own class- 
room practice. For more information on 
EFF, check out http: / / www.nifl.gov / eff. 
or call 1-877-433-7827. 

In What Makes a Good Teacher? Marie 
Hassett identifies eight characteristics of a 
good teacher. She hopes these will be help- 
ful as we look at and reflect on our own 
teaching and connection to our students. 

Caroline Gear, Rebecca Shiffren and 
Steve Kurtz write about the importance of 
empowering teachers to problem solve 
about classroom issues. They introduce a 
tool that was developed at the Center for 
Teacher Education at The School for Inter- 
national Training in Brattleboro, Vermont . 
This tool helped them establish a way to 
give and receive feedback that involved 
the teacher more in the process. 

In Learning from Experience, Bemie 
Driscoll from the Taunton Adult Learning 
Center shares her math assessment process 
and tool and talks about the difficulty 
students may have when faced with an 
assessment tool that is too daunting. 

Many teachers are looking at standards- 
based assessment and what it means for 
them and their classrooms. As the initia- 
tives grow in use, it would be interesting 
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to hear what other teachers are doing to 
translate these initiatives into practice. 
What does it mean to develop assessment 
tools, to write assessment criteria? To de- 
velop performance standards? How are 
we using the EFF standards in our class- 
rooms? How are we using the other frame- 



works to better understand what it is our 
students know, need to know and be able 
to do? 

As always we welcome your thoughts 
and ideas. If you would like to submit an 
article or have comments, please feel free 
to contact me at asimmons@worlded.org . 
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What Makes A Good Teacher? 



I have been teaching for the last ten 
years. During that time, I have worked 
in public schools, universities, extracur- 
ricular programs for K-12, adult basic lit- 
eracy, and adult enrichment classes. My 
youngest student was a 6 year-old bud- 
ding actress in a town-sponsored arts en- 
richment program for elementary students; 
my oldest, a Jamaican immigrant, a grand- 
mother beginning at the age of 63 to learn 
how to read. I've taught honors students in 
a college humanities program, and se- 
verely handicapped youth in a public high 
school. 

The breadth of my experience has en- 
riched my teaching life, but left me with- 
out a luxury some of my colleagues 
enjoy — the sense, as I walk into a new 
class, for a new term, that I know what my 
students will need, and how best to share 
it with them. This is not to say that I've 
been tossed blind into the classroom. In 
most cases. I've had enough prep time to 
gather what seem like appropriate materi- 
als, and find out something about the stu- 
dents I'll be working with. What I have not 
had is the critical mass of sameness that ac- 
crues to the teacher who stays in the same 
setting, at the same level, for many years in 
a row. I cannot assume that what worked 
last semester will work this time. 

As a result of my ever-changing context. 
I've spent a lot of time thinking about the 
craft and practice of teaching, as separate 
from course content, age of students, size 
of class, or institutional setting. Every- 
where I go, I meet exemplary teachers, and 

O 



I've been interested in figuring out what 
makes them so good. What I've discov- 
ered is the inherent sameness of good 
teachers, regardless of the substantial dif- 
ferences between them in terms of style, 
personality, goals, and pattern of interac- 
tion with students. I would go so far as to 
say that good teachers, in all settings and 
at all levels, have more in common with 
each other than any of them may have 
with their colleagues in comparable posi- 
tions. 

In order to understand the bold state- 
ment above, try the following exercise. Sit 
back, close your eyes, and bring to mind 
the three best teachers you ever had. Try 
to remember what they were like — how 
they looked, talked and acted, what their 
classrooms and/ or offices were like, how 
they made you feel as their student. When 
you're satisfied that you've gotten a good 
picture of who these people were, open 
your eyes, and consider the words of edu- 
cator and philosopher Parker Palmer: 

Good teaching isn't about technique. I've 
asked students around the country to de- 
scribe their good teachers to me. Some of 
them describe people who lecture all the 
time, some of them describe people who do 
little other than facilitate group process, and 
others describe everything in between. But 
all cf them describe people who have some 
sort of connective capacity, who connect 
themselves to their students, their students 
to each other, and everyone to the subject 
being studied. (1999, p. 27) 
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If you want 
your students to 
become better, 
more involved 
readers, you allow 
time for reading 
and provide access 
to books. 



Do you recognize your best teachers in 
this description? When we talk about the 
quality of someone's teaching, we address 
issues of technique, content, and presenta- 
tion. But we all know people who have 
tremendous knowledge but fail to commu- 
nicate it: people who have, on paper, a 
great lesson, but whose students are bored 
or frustrated. When we're being honest, we 
admit that good teaching often has less to 
do with our knowledge and skills than 
with our attitude towards our students, 
our subject, and our work. 

The rest of this article will address some 
of the characteristics that good teachers ex- 
hibit. It is not meant to be all encompass- 
ing or definitive; many excellent teachers 
may possess only some of these traits, and 
consider others not mentioned to be just as 
valuable. The characteristics detailed here 
may be viewed simply as a selection of 
tools that allow teachers to create and sus- 
tain connectivity in their classrooms. 



Good teachers: 

• have a sense of purpose; 

• have expectations of success for all stu- 
dents; 

• tolerate ambiguity; 

• demonstrate a willingness to adapt and 
change to meet student needs; 

• are comfortable with not knowing; 

• reflect on their work; 

• learn from a variety of models; 

• enjoy their work and their students. 



GOOD TEACHERS HAVE 
A SENSE OF PURPOSE. 

You can't be good in a generic sense; you 
have to be good for something. As a 
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teacher, this means that you know what 
your students expect, and you make plans 
to meet those expectations. You, too, have 
expectations about what happens in your 
classroom, based on the goals you're try- 
ing to achieve. If you want to prepare your 
students for employment, you expect 
punctuality and good attendance. If you 
teach a GED class, you spend time explain- 
ing the format of the test and helping stu- 
dents to improve their test-taking skills. 
And if you want your students to become 
better, more involved readers, you allow 
time for reading and provide access to 
books. 

GOOD TEACHERS HAVE EXPECTATIONS 
OF SUCCESS FOR ALL STUDENTS. 

This is the great paradox of teaching. If we 
base our self-evaluation purely on the suc- 
cess of our students, we'll be disappointed. 
At all levels, but especially in adult educa- 
tion, there are simply too many factors in 
students' lives for a teacher to be able to 
guarantee success to all. At the same time, 
if we give up on our students, adopting a 
fatalistic, "it's out of my hands" attitude, 
students will sense our lack of commit- 
ment and time out. The happy medium 
can be achieved with a simple question: 
Did I do everything that I could in this 
class, this time, to meet the needs of all my 
students, assuming that complete success 
was possible? As long as you can answer 
in the affirmative, you're creating a climate 
for success. 

GOOD TEACHERS KNOW HOW 
TO LIVE WITH AMBIGUITY. 

One of the greatest challenges of teaching 
stems from the lack of immediate, accurate 
feedback. The student who walks out of 
your classroom tonight shaking his head 
and muttering under his breath about alge- 
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bra may burst into class tomorrow pro- 
claiming his triumph over math, and 
thanking you for the previous lesson. 

There is no way to predict precisely what 
the long-term results of our work will be. 
But if we have a sense of purpose inform- 
ing our choice of strategies and materials, 
and we try to cultivate expectations of suc- 
cess for all our students, we will be less 
likely to dwell on that unpredictability, 
choosing instead to focus on what we can 
control, and trusting that thoughtful 
preparation makes good outcomes more 
likely than bad ones. 

GOOD TEACHERS ADAPT AND CHANGE 
TO MEET STUDENT NEEDS. 

Can we really claim to have taught a class 
in geography if no one learned any of the 
concepts in the lesson from our presenta- 
tion? If none of our students ever pick up a 
book outside of the classroom, have we re- 
ally taught them to be better readers? We 
don't always think about these issues, but 
they are at the heart of effective teaching. 

A great lesson plan and a great lesson are 
two entirely different things; it's nice when 
one follows the other, but we all know that 
it doesn't always work out that way. We 
teach so that students will learn, and when 
learning doesn't happen, we need to be 
willing to devise new strategies, think in 
new ways, and generally do anything pos- 
sible to revive the learning process. It's 
wonderful to have a good methodology, 
but it's better to have students engaged in 
good learning. 

GOOD TEACHERS ARE REFLECTIVE. 

This may be the only infallible, absolute 
characteristic of all good teachers, because 
without it, none of the other traits we've 
discussed can fully mature. Good teachers 
routinely think about and reflect on their 
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classes, their students, their methods, and 
their materials. They compare and con- 
trast, draw parallels and distinctions, re- 
view, remove and restore. Failing to 
observe what happens in our classes on a 
daily basis disconnects us from the teach- 
ing and learning process, because it's im- 
possible to create connectivity if you've 
disconnected yourself. 

GOOD TEACHERS ARE COMFORTABLE 
WITH NOT KNOWING. 

If we reflect honestly and thoughtfully on 
what happens in our classes, we will often 
find dilemmas we cannot immediately re- 
solve, questions we cannot answer. In his 
Letters to a Young Poet, Rainer Maria Rilke 
suggests that his correspondent, "try to 
love the questions themselves as if they were 
locked rooms or books written in a very 
foreign language.... Live the questions 
now. Perhaps then, someday far in the fu- 
ture, you will gradually, without even no- 
ticing it, live your way into the answer" 
(1986, pp. 34-35). In the same way, our 
teaching benefits if we can live for a little 
while with a question, think and observe, 
and let an answer develop in response to 
the specific situation we face. 

GOOD TEACHERS HAD 
GOOD ROLE MODELS. 

Think back again to your three best teach- 
ers. How has your own teaching been 
shaped by their practices, consciously or 
unconsciously? Think also of the worst 
teacher you ever had. Are there things you 
absolutely will not do because you remem- 
ber how devastating they were to you or 
your classmates? We learn to teach gradu- 
ally, and absorb ideas and practices from a 
variety of sources. How many movies have 
you seen that include a teacher as a charac- 
ter, and how might those films have con- 



If we reflect 
honestly and 
thoughtfully 
on what happens 
in our classes, 
we will often 
find dilemmas 
we cannot 
immediately 
resolve. 
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tributed to your practice? We are not al- 
ways aware of the influences on our teach- 
ing, good and bad; reflecting on the 
different models of teaching we've ac- 
quired, and looking at how we acquired 
them, makes us better able to adapt and 
change to suit new challenges. 

GOOD TEACHERS ENJOY THEIR WORK 
AND THEIR STUDENTS. 

This may seem obvious, but it's easy to 
lose sight of its importance. Teachers who 
enjoy their work and their students are 
motivated, energized, and creative. The 
opposite of enjoyment is burnout — the 
state where no one and nothing can spark 
any interest. Notice, too, that enjoying 
your work and enjoying your students 
may be two different things. Focusing too 
much on content may make students feel 
extraneous, misunderstood, or left out. Fo- 
cusing exclusively on students, without an 
eye to content, may make students feel un- 
derstood and appreciated, but may not 
help them to achieve their educational 
goals as quickly as they'd like. Achieving a 
balance between the two extremes takes 
time and attention; it demands that we ob- 
serve closely, evaluate carefully, and act on 
our findings. 

I would like to conclude with a poem by 
Lao-Tzu, the Chinese scholar to whom the 
Tao Te Ching is attributed. I have carried a 
copy of this poem with me for many years, 
and I find its message both helpful and 
challenging. It reminds us that good teach- 



ing is not a static state, but a constant 
process. We have new opportunities to 
become better teachers every day; good 
teachers are the ones who seize more 
opportunities than they miss. 

Some say that my teaching is nonsense. 

Others call it lofty but impractical. 

But to those who have looked inside themselves, 
this nonsense makes perfect sense. 

And to those who put it into practice, 
this loftiness has roots that go deep. 

I have just three things to teach: 
simplicity, patience, compassion. 

Simple in actions and thoughts, 
you return to the source of being. 

Patient with both friends and enemies, 
you accord with the way things are. 
Compassionate toward yourself, 

You reconcile all being in the world. (1989, 17) 

Works Cited 
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Successful Supervision: Three Perspectives 



I have always looked for the perfect tool 
to use when observing teachers, and I 
have tried many different ones. What 
has become clear to me in the past year is 
that what matters most is not so much 
finding the "perfect tool", but how one 
uses it and adapts it to one's own program 
and purpose for observation. 

For years I would take copious notes 
while observing a teacher and afterwards, I 
would type up the notes into a report form 
divided into the three categories of 1) time 
of activity, 2) description of the activity 
and 3) general comments. My biggest com- 
plaint in conducting observations in this 
way was that it was extremely time con- 
suming. I spent more time writing the re- 
port then I did discussing the class with 
the teacher. After the follow-up meetings 
I would wonder how effective they really 
were. Were teachers using the observations 
and follow-up meetings as ways to im- 
prove their teaching? Or was this just a 
tool for me to evaluate teachers in the 
classroom and all the teachers got out of it 
was an evaluation in their file? As I look 
back at the way I used to conduct evalua- 
tions, I realize the process was too one- 
sided, not allowing a lot of teacher 
reflection. 

My observation and follow-up with 
teachers changed dramatically in January 
of 1999 when I became involved in the 
School for International Training's (SIT) 
TESOL Certificate Program. In partnership 
with SIT, the International Language Insti- 
tute now provides both intensive and part- 
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time TESOL Certificate Programs. 

An integral part of the SIT TESOL Cer- 
tificate Program is giving the participants 
in the program a hands-on opportunity to 
teach followed by an in-depth feedback on 
the class. The form that SIT uses for its 
teacher observations is simple (see at- 
tached form). I liked the form immediately. 
It takes the three categories that I was us- 
ing in my previous observations and puts 
them in a format that is clear. Most impor- 
tantly, in the "comments" category, the 
emphasis is on posing questions rather 
than writing possibly critical statements on 
what is going on in the class. 

After using this assessment tool a few 
times, I began to realize that how I had 
previously been observing teachers was 
more "observer centered". My observa- 
tions and follow-up meetings were not set 
up to allow the teachers time to reflect on 
the class that they had taught, and I was 
the one who would initiate the discussion 
about the class. 

Under the SIT model, I now give the 
instructor a copy of the form I filled out 
while observing the class as soon as the 
class is over. We then make an appoint- 
ment within the next few days to discuss 
the class. Teachers have the opportunity 
between the observation and the follow-up 
meetings to really think about specific 
questions or issues, rather than wonder 
what comments the observer had about 
the class. Discussions during the follow-up 
meetings are initiated more by the teacher 
as s/he responds to the questions that 
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The best follow-up 
meetings are when 
there is truly a 
dialogue between 
observer and 
teacher. The play- 
ing field has been 
leveled and both 
parties strategize 
together on ways to 
improve the class. 



were posed to her/him on the observation 
form rather than a one-sided conversation 
with the observer telling the teacher about 
what worked or did not work in the class- 
room. 

I do not think any assessment tool 
works perfectly the first time that it is 
used. When using the SIT Observation 
Form, the more the observer becomes fa- 
miliar with the process of asking the right 
questions for teacher reflection, the better 
the follow-up meetings will be. For me, the 
best follow-up meetings are when there is 
truly a dialogue between observer and 
teacher. The playing field has been leveled 
and both parties strategize together on 
ways to improve the class. 

Obviously, this type of dialogue can 
never happen if the observations are 
handled by administrators who have very 
little teaching experience and whose pur- 
pose is only to evaluate the teacher. This 
assessment tool works best when the ob- 
server has a lot of classroom instruction 
experience along with the ability of high- 
lighting the strengths and posing questions 
when there are weaknesses. The purpose 
of the observation tool is focused more on 
giving the teacher support and providing 
an arena for reflection and growth rather 
than just evaluating her/his teaching. 

Just as I was changing my method of ob- 
serving teachers, Rebecca Schiffren from 
Lutheran Social Services called me with 
concerns about her observations of teach- 
ers. The International Language Institute 
and Lutheran Social Services of West 
Springfield are funded together to provide 
ESOL services both in Northampton and 
West Springfield. 

Rebecca and I met to discuss the tool 
and then both used the tool to observe a 
teacher who was teaching a practice class. 
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(As part of the hiring process at ILI, appli- 
cants have to teach a 30-minute practice 
class.) We found that the assessment tool 
was difficult to use in a practice class, but 
it allowed Rebecca and me to talk about 
what questions we would have asked the 
teacher about the class and a hands-on 
opportunity to evaluate how this tool 
would work with other teacher trainers. 

What follows is how Rebecca used the 
tool with one of her teachers, Steve Kurtz, 
and Steve's reaction to the observation and 
follow-up meeting. 

Rebecca's piece: 

I was hired as an ESL teacher at 
Lutheran Social Services (LSS) in 
September, 1995. (I had been teaching 
ESL for six years before that.) In March, 
1999 1 was given the position of ESL Coor- 
dinator and part of my job was to observe 
and coach other teachers in our program. 

I had never done this before and began 
by using a format that had been helpful to 
me when a supervisor used it while ob- 
serving my class the previous year. That 
method was to describe the activities in de- 
tail and give positive feedback and con- 
structive suggestions as to how to improve 
the lesson. 

When I used this approach with one 
relatively new teacher, she said the critique 
and suggestions were helpful. With an- 
other teacher, however, the technique was 
not so successful. This was a relatively in- 
experienced teacher whose class atten- 
dance was slipping. I felt, after observing 
him, that there was a lot of room for im- 
provement. I made many suggestions but 
sensed as the critique went on, that he be- 
came more and more defensive. In the end, 
I wasn't convinced he would be able to 
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take in the feedback and improve his 
teaching. 

At this point, I met with Caroline Gear, 
who introduced me to a new technique she 
had learned working with The School for 
International Training (SIT). This approach 
was to use three columns: one for the time, 
the next for a running account of what was 
happening in the class, and the third for 
comments posed as questions. At the end 
of the lesson, the observer would pose 
more general questions that considered the 
lesson as a whole. The idea was for the in- 
put not to be critical but to give teachers 
room to think about ways of solving prob- 
lems that made sense to them. I wondered 
whether this would work since raising a 
question, in my mind, implied a criticism. 

Coincidentally, the meeting with 
Caroline happened just before my next 
scheduled observation of Steve so I had a 
perfect place to try out the new method. 
When students hadn't written in journals 
as assigned, I wrote, "Why do you think 
they aren't writing?" "How can you get 
them more interested in writing?" When 
students had trouble remembering and 
pronouncing past tense verb forms, I 
asked, "What other ways could you prac- 
tice irregular past tense forms?" In the 
review session after the lesson, Steve 
reflected on these and other questions 
and came up with some new ways of 
approaching problems. When he was fin- 
ished thinking about a question, I shared 
some of my experiences with the same 
issue. I felt that Steve was much more in- 
volved in this session. He thought about 
his teaching and was more enthusiastic 
and less defensive. 

When I observed him again about a 
month later, he had made real strides in 
his teaching. He was much more assertive, 
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had a plan and followed it. His students' 
attendance during the month had notice- 
ably improved and they gave me positive 
feedback about his teaching (as opposed to 
earlier, when all I heard were complaints). 

This method of observation and coach- 
ing proved to be extremely successful for 
Steve. It encourages reflection and explora- 
tion that is meaningful and empowering 
because it comes from the teacher's own 
experience. The method does require skill 
on the part of the observer — you must be 
aware of classroom dynamics and pose 
meaningful questions — but it gives teach- 
ers the responsibility (and power) to be ac- 
tively involved in their own development. 

Steve's piece: 

I have worked for twelve years as a 
teacher. Before my current position as 
an ESOL instructor, I was in the public 
schools, initially as a Spanish teacher and 
then as a bilingual social studies teacher. I 
have been in my current position as an 
ESOL instructor for two years. 

Throughout my teaching career; super- 
visors have observed me between eight 
and ten times. In my current job, I have 
been observed on four different occasions 
by supervisors. I have made progress in 
improving my teaching style and class- 
room management; however my progress 
has been rather slow for the better part of a 
year. I struggled with important teaching 
areas and I began to wonder about my 
suitability for the profession. 

My current supervisor began her official 
duties approximately six months ago. She 
has observed me four times, the first obser- 
vation having occurred on April 14, 1999 
and the most recent, on October 13, 1999. 
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My feelings during 
both the first and 
second classroom 
observations were 
similar: anxiety, 
nervousness, 
awkwardness, 
embarrassment, 
and increased 
adrenaline. 
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Each of the observations consisted of an 
observation of me in the classroom and a 
feedback session, which followed. In this 
paper I want to describe how I have expe- 
rienced these sessions and how the 
changes in my supervisor's feedback ap- 
proach have impacted on my teaching. 

After the first observation my supervi- 
sor and I met. She shared her notes and we 
discussed the lesson. Her notes consisted 
of six comments. For each comment she 
had written suggestions for ways to im- 
prove or modify my teaching. I felt some- 
what deflated and discouraged during the 
above-described meeting. I listened to my 
supervisor, tried to act like a good adult 
professional, yet I felt like very much the 
opposite. My feelings of discouragement 
continued for a while after the session. I 
knew the suggestions were good. Now I 
had to "deliver." 

Before the second observation/feedback 
session (July 7, 1999) my supervisor in- 
formed me that she was going to change 
her approach. She briefly described the 
new approach and how it would differ 
from the first observation in April. She 
asked me if I wanted her to focus on any 
particular aspect of my teaching. 

In the July 7 feedback session my super- 
visor included clock times for her observa- 
tions. After each comment she posed a 
question. For example: 



Comment: 6:25 Students talked about 
fireworks, not seeing them before. 

Question: "How could you have ex- 
tended the talk about fireworks? They 
were interested and it was 'real' talk." 

I then offered my response to the ques- 
tion. "I could have encouraged them to 
talk about fireworks in their own countries 
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and how they compare with fireworks in 
the USA." 

During the July 7 class I had planned a 
session in which students would practice 
question formation using the July 4 week- 
end as a stimulus. Students were very in- 
volved in discussing the fireworks theme 
in their conversation partners, when I 
ended this activity and began another ac- 
tivity involving story writing with num- 
bered pictures. The abrupt move from an 
activity in which students were very en- 
gaged to another activity with little transi- 
tion or preparation confused the students. 
Moreover, this new activity was unrelated 
to its predecessor. At the end of this ses- 
sion (and subsequent sessions) my super- 
visor included comments which were 
more globed in scope. These comments 
helped me come away from the session 
with a kind of quiet "mantra," which 
would apply to a broader range of situa- 
tions. 

Also, in the second and subsequent 
feedback sessions, my supervisor posed 
her questions and waited for my response. 
This waiting or pause was an invitation for 
me to offer a thoughtful response. I was 
given real time. My response was impor- 
tant. 

My feelings during both the first and 
second classroom observations were simi- 
lar: anxiety, nervousness, awkwardness, 
embarrassment and increased adrenaline. 

During the third and fourth observa- 
tions, I continued to be anxious and felt 
high adrenaline flowing through me. The 
difference was that I felt more energetic, 
more in control and, in general, more posi- 
tive about my teaching. I'm not certain 
how much of this improvement can be at- 
tributed to the new observation proce- 
dures. Perhaps, the good results have 
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come from both the new procedures and 
from the increased trust I have for my su- 
pervisor. 

During this process my feelings about 
my work have changed. I feel more hope- 
ful and more positive. I am still very much 
a teacher who needs to improve his perfor- 
mance. The difference is in the kind of clar- 
ity of thought that I have now. "Clarity" 
for me means that I'm focusing more on 
what I am doing and less on what I'm fail- 
ing to do. The question is "Why?" 

First, by presenting information and ob- 
servations and following each observation 



with a question, my supervisor is inviting 
me to participate. She is implying that I 
have the ability to be analytical. The ques- 
tion is still a form of criticism but it is a 
form of criticism in which I offer my own 
analysis. The analysis is mine. 

Second, the process of observation and 
question invites me to focus on a solution. 
Since I am invited to answer a question, to 
respond to that question, I am further em- 
powered to design a solution and imple- 
ment it. I am focusing on what I need to do 
to improve and not on what I haven't done 
or on what I have been failing to do. 



& 



Teacher: 



SIT TESOL Certificate Program Observation Sheet 



Date: 



Number of Students: Level: 



Observer: 



Time 



Steps of Lesson 



Comments 
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A Curriculum Project 



by 

Sherry Spaulding 




International Institute 
of Boston 



T he International Institute of Boston 
(IIB) has been in operation serving 
refugees and immigrants in the Bos- 
ton community for more than 75 years. 
Through the years the agency has offered a 
variety of educational programs to refu- 
gees and immigrants and the opportunity 
for many talented teachers to develop 
well-planned, well-written curricula. As 
changes occur in student population, 
teaching philosophies and methodologies, 
learners' needs, and in our language and 
culture, it is necessary for curricula to con- 
tinually evolve. 

Earlier this year, IIB received funding 
from the Massachusetts Department of 
Education for a six-month project to ex- 
pand the current curriculum for our 
evening ESOL program. This program cur- 
rently offers three levels of ESOL to adult 
immigrants and refugees. Classes are held 
two evenings a week with an optional 
third evening for learning computer skills 
and for computer-assisted language learn- 
ing (CALL). The proposal we submitted 
for this project outlined three objectives: 1) 
to develop an evening ESOL curriculum; 2) 
to develop sample needs assessment tools; 
and 3) to incorporate technology into the 
ESOL curriculum, all in all, a very ambi- 
tious proposal. The majority of the work I 
did as coordinator of this project was on 
the first two objectives. The third objective 
was accomplished with the help of our 
Technology Coordinator who trained 
teachers one-on-one and helped them to 
develop appropriate CALL activities for 
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their classes. 

The curriculum which was being used 
in IIB's evening ESOL program prior to 
this project had been based primarily on 
life skills and job skills because it was de- 
veloped for our intensive day program for 
newly-arrived refugees and dislocated 
workers. Because our evening ESOL pro- 
gram serves a greater diversity of learners 
from many different countries — refugees, 
immigrants, employed, unemployed — it 
was necessary to develop a curriculum 
that would address a broader range of 
needs. I wanted to take a bottom-up ap- 
proach to developing this curriculum by 
assessing both learners' and teachers' 
needs and utilizing the knowledge and ex- 
pertise that our teachers bring to the pro- 
gram to develop a helpful and informative 
curriculum guide. I also worked with a 
group of Massachusetts teachers. Depart- 
ment of Education representatives and 
consultants who have been working to- 
gether to develop a statewide Framework 
for Adult ESOL. 

GATHERING LEARNER INPUT 

Because I stepped out of my role as teacher 
to be the coordinator of this project, I de- 
cided to work primarily with the ESOL 
members of our Student Council. The 
Council consists of two representatives 
from each of our evening ESOL and ABE 
(for non-native English speakers) classes. 
The Council was originally formed to have 
more student representation in planning 
and evaluating our evening programs, and 
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to provide a vehicle for tapping into our 
learners' needs. It also gave those partici- 
pating on the Council an opportunity to 
take on leadership roles: offering ideas for 
improving our programs, planning events, 
informing fellow classmates of events, de- 
veloping surveys and gathering input from 
fellow classmates. 

I met with the ESOL members of our 
Student Council for about 30-45 minutes 
prior to class time on three occasions. The 
meetings were conducted in English. I 
posed several questions, starting with the 
obvious: "Why are you here? Why do you 
need to learn English?" The obvious re- 
sponse was to become self sufficient, or in 
the words of one Council member, "7 don't 
like to ask my friends, my family, please help 
me." The questions I generated focused on 
the use of English at home, at work, and in 
the community. 

I chose to focus on these three areas be- 
cause of information I have gathered at 
meetings and conferences; from speakers 
from Equipped For the Future (EFF); from 
Heide Spruck Wrigley, a consultant from 
AGUIRRE International who often dis- 
cusses the importance of gathering input 
on learners' needs in the areas of home, 
work, neighborhood /community, educa- 
tion/school and getting things done; and 
from representatives from the Massachu- 
setts Department of Education who em- 
phasized at an initial curriculum 
frameworks meeting the need for a cus- 
tomer-driven learning environment that is 
responsive to the needs of learners in their 
three roles: as family member, worker and 
citizen. 

My hope was to obtain information 
from learners themselves about their 
needs, interests and goals. With that infor- 
mation, we could create a learner-centered 
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curriculum. How to gather information 
from students with limited English lan- 
guage skills is always a challenge. Often I 
used pictures to facilitate conversation. For 
example, to initiate our discussion about 
work, I used pictures of people in various 
jobs as a code for getting students to talk 
about their own work situations. The ques- 
tions I asked in the three areas included: 

At Home: Why do you need English at 
home? How do you feel when the phone 
rings? Who do you need to talk to on the 
phone in English? What mail do you need 
to read in English? What do you do when 
you can't read the mail you receive? Do 
your children often talk on the phone and 
translate mail for you? How do you feel 
about that? 

At Work: Was it easy to get your job? 

Why/ why not? How did you get your job? 
Do you like your job? Why/why not? 

What did you do in your native country? 
Do you need English for your job? Why/ 
why not? Who do you talk to? What do 
you need to read at work? Write at work? 
What are your plans for the future? 

In the Community: Where do you go 
alone? Where do you go with someone 
else? Why do you need someone else to go 
with you to this place? Are there places 
you don't go? Why? Do you have prob- 
lems that you need to solve but you don't 
know where to go for the right informa- 
tion? 

We gathered considerable input on 
what systems learners need to navigate 
when the Coordinator of Adult Education 
and I met with all the Student Council 
members from our ABE and ESOL pro- 
grams to brainstorm on which guest 
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It seemed impos- 
sible to generate 
one survey that 
was appropriate for 
all the skill levels of 
our learners in 
ESOL and ABE. 



speakers they would like to invite to our 
school. The Council gave ideas and shared 
personal stories and then we surveyed all 
of the learners in the two programs. 

It seemed impossible to generate one 
survey that was appropriate for all the skill 
levels of our learners in ESOL and ABE, so 
I developed three versions of a survey in 
hopes that each teacher could use one ap- 
propriate to his/her class and as part of a 
lesson. For example, the survey used by 
the lowest level ABE class displays pic- 
tures to help represent each system. The 
teacher could use the pictures on the sur- 
vey to brainstorm on large paper reasons 
students gave for wanting a certain 
speaker and the survey was simple enough 
for these learners to check which speakers 
they wanted. 

Another version asked learners "why" 
they wanted a certain speaker, which gave 
them an opportunity to write. Also, rather 
than asking students directly, "What prob- 
lems do you have?", this more indirect 
way of simply asking students why they 
are interested in this guest speaker encour- 
ages them to share personal stories if they 
wish without feeling anxious about having 
to tell their problems. The feedback we 
gathered from this particular activity was 
very informative and was used to develop 
the section in our ESOL curriculum on 
navigating systems. 

The surveys are included at the end of 
this article. 



GATHERING TEACHER INPUT 

I met several times with teachers in staff 
meetings and one-on-one. I wanted to 
know how often they referred to the cur- 
rent curriculum guide, what they thought 
should be included in a curriculum and 
what was not clear to them about our 



ESOL program. 

Because there had been some confusion 
as to when to move students to a higher or 
lower level class, it seemed obvious that 
our levels needed to be more clearly de- 
fined for teachers. To address this, I asked 
teachers to list their students' strengths 
and weaknesses on large pieces of paper in 
the categories of the five strands of the 
Frameworks for Adult ESOL: oral and 
written communication, language struc- 
ture and mechanics, navigating systems, 
intercultural knowledge and skills, and de- 
veloping strategies and resources for learn- 
ing. 

The first significant piece of information 
we acquired from this activity was that the 
teachers could very easily list specific 
strengths and weaknesses in only two of 
the five strands: oral and written commu- 
nication, and language structure and me- 
chanics. The reason was primarily because 
these are the areas we assess. At the end of 
each term our teachers meet with learners 
one-on-one to assess speaking and listen- 
ing skills and then learners are given a 
written test to assess reading and writing 
skills. Therefore, we are assessing a 
learner's ability to communicate meaning- 
fully and accurately. 

It also became very clear to us through 
this activity that the primary focus of our 
Beginner Level class is on developing oral 
communication skills. In Level 1 the learn- 
ers' needs are more varied in terms of oral 
and written communication skills but the 
focus is still on improving oral communi- 
cation. In Level 2, learners have the skills 
to communicate; they need to work on ac- 
curacy. Because our program places learn- 
ers in classes based primarily on listening/ 
speaking ability rather than reading/writ- 
ing ability, this is where there is the great- 
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est differentiation of skill areas. Some 
learners are very weak in reading and/ or 
writing while others who have studied in 
our program or other ESOL programs are 
more proficient in those areas. 

To tap into the teachers' knowledge and 
skills, I asked them how they decide what 
to teach, how they assess their learners' 
needs, and how they involve students in 
curriculum planning. Much of the input I 
gathered from these meetings with teach- 
ers has been included in a section of our 
curriculum on ways to assess learners 
needs. I also developed a needs assessment 
resource binder which includes sample les- 
sons and activities teachers have used to 
gather input on learners' interests, needs 
and goals. 

EVENING ESOL CURRICULUM 

As a result of many productive meetings 
and helpful input from active learners and 
experienced teachers, an evening ESOL 
curriculum with a number of sections 
emerged. We hope it will give teachers a 
holistic picture of the learners, program, 
curriculum content, important definitions 
and procedures specific to IIB. 

Defining our program 
In order to give teachers a clear under- 
standing of our program and how learners 
are placed in levels we included sections 
entitled: 

• What is IIB's mission? 

• Who are our ESOL learners? 

• The mission of our evening ESOL pro- 
gram 

• How do we initially assess learners? 

For example, the way in which we as- 
sess and place learners into classes is as fol- 
lows: 
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• Applicants are asked to complete an 
intake form as best they can without as- 
sistance in order to check for basic com- 
prehension of forms. The intake form 
asks for personal information, work in- 
formation and personal goals. 

• A teacher will begin with an informal 
conversation with the applicant to pre- 
determine the range of level, and to se- 
lect appropriate assessment tools as well 
as to help the applicant feel at ease dur- 
ing the assessment. 

• The teacher gives the short form of the 
BEST oral test to the applicant to deter- 
mine an SPL level for listening and 
speaking. 



• Applicants are then asked to read a pas- 
sage at one of three levels which is cho- 
sen by the teacher based on the BEST 
score. The teacher asks questions about 
the passage orally to which the learner 
responds orally. The teacher will choose 
an appropriate grammar test or tests 
from three examples. Finally, the appli- 
cant is asked to write responses to ques- 
tions or to write a short paragraph. 
Again the teacher chooses which format 
is most appropriate to the applicant, in 
order to check vocabulary, spelling and 
grammar. The teacher then determines 
an SPL level for reading and writing 
based on the reading passage, grammar 
test(s) and writing sample. 



• The teacher decides level of placement 
based primarily on the SPL for speaking 
and listening but uses the reading/writ- 
ing score to determine whether the ap- 
plicant has formally studied English 
previously, which may promote him/ 
her to a higher level. Also taken into 
consideration is an applicant's level of 
education and whether he/ she has con- 
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tact with other native English speakers 
outside the class. If an applicant has 
skills which are too high for our pro- 
gram, he/she is provided information 
on other programs. 

• If an applicant's literacy level is too low 
for our ESOL program, several options 
are considered. If the applicant has an 
SPL 3 or higher in speaking and listen- 
ing he/ she may be a candidate for our 
ABE for non-native English speakers 
program which focuses on literacy, 
reading and writing. If the applicant is a 
beginner in speaking, listening and lit- 
eracy, he/she has the option of joining a 
daytime Beginner ESOL Basic Literacy 
class if his/her schedule allows. IIB also 
provides volunteer trainings and appli- 
cants may be paired with a volunteer tu- 
tor. 

Defining Our Levels 
In order to more clearly define our three 
ESOL levels and learners, we incorporated 
a detailed description of each level includ- 
ing the following sections: 

• SPLs 

• Student profiles 

• Expected outcomes 

• Curriculum guide to content areas 

• Language structure and mechanics 

The SPL section describes the listening/ 
speaking and reading/writing range of 
learners entering that level. The profiles 
provide general descriptions of learners at 
that level which may emphasize inconsis- 
tencies, e.g., in listening comprehension, 
pronunciation, reading or writing abilities. 
The section on expected outcomes de- 
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Determining the 
expected outcomes 
for each level was 
the most difficult 
part to write. 



scribes the primary focus of learning at 
that level and what should be achieved 
prior to moving to the next level. Finally, 
the guides to content areas and language 
structure and mechanics are lists of life 
skills and grammar skills to be addressed 
at that level. 

Determining the expected outcomes for 
each level was the most difficult part to 
write. Our outcomes are general and yet 
they do reflect what learners should be 
able to do before entering a higher level. 
Because class placement is primarily based 
on learners' oral and listening comprehen- 
sion skills, our outcomes focus more on 
improvement of these skills and less on 
reading and writing skills. For Beginner 
Level and Level 1 there is a stronger em- 
phasis on ability to communicate, while 
Level 2 emphasizes accuracy in ability to 
communicate. Below are a few examples of 
outcomes developed for each ESOL level. 

Beginner Level 

• Learners are able to respond to basic 
oral and written questions about per- 
sonal information including name, ad- 
dress, telephone number, age, date of 
birth, and country of origin. 

• Learners are able to respond to basic 
oral and written questions about their 
current job situation or past job(s), for 
example, job title(s), place of employ- 
ment, at least one duty, and dates of em- 
ployment. 

Level 1 

• Learners are able to respond to oral and 
written questions about personal infor- 
mation including name, address, tele- 
phone number, social security number, 
age, sex, marital status, date and coun- 
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try of birth, nationality, and native 
language(s). 

• Learners are able to communicate their 
future plans and personal goals. 

Level 2 

• Learners are able to monitor their own 
speech, meaning that they are self-cor- 
recting or can self-correct when an error 
has been made apparent to them, pro- 
vided the grammar structure has been 
taught at this level. 

• Learners are able to clearly communi- 
cate their problems in contexts which af- 
fect their lives, e.g., housing problems, 
health problems, etc. 

Other Helpful Sections in our 
ESOL Curriculum 

In addition to the sections in IIB's curricu- 
lum defining our program and levels, 
we've included several other sections 
which include: 

• Navigating systems 

• Intercultural knowledge and skills 

• Developing strategies and resources for 
learning 

• Ways to assess learners' needs 

• Guidelines for cycle-end assessment of 
students 

• Guidelines for determining SPLs 

• M.E.L.T. student performance levels 

As we heard from students, it was clear to 
us that understanding and negotiating a 
complexity of systems was important. We 
included a section entitled "Navigating 
Systems" in our curriculum. It is similar to 



the guide to content areas incorporated 
into the descriptions of levels, but is di- 
vided by system as opposed to level of 
proficiency. Here is an example of one 
system: 

Systems of Transportation 

• Knowing types of public transportation, 
e.g., subways, trains, buses, and com- 
muter rail. 

• Being able to ask for directions and loca- 
tion of stops, to request a stop on a bus 
or subway. 

• Knowing where to get schedules, how 
to ask for and read schedules. 

• Knowing where to buy tokens and 
passes and the different types of passes. 

• Being able to ask for a certain amount of 
tokens or a certain type of pass. 

• Knowing how to get to destinations be- 
yond the local area using transportation, 
e.g., Amtrak, airlines or bus lines. 

• Knowing how to buy tickets and make 
reservations to places beyond the local 
area, and how to ask questions regard- 
ing schedules and prices. 

The sections on "Intercultural Knowledge 
and Skills" and "Developing Strategies and 
Resources for Learning" primarily make 
reference to resources at IIB and the Mas- 
sachusetts Frameworks for Adult ESOL for 
further information. To help teachers plan 
ways to assess their learners' needs, we in- 
cluded in our curriculum a collection of 
methods gathered from teachers and from 
reference materials for assessing what 
learners need and want to learn. To help 
teachers plan final evaluation activities we 
have included guidelines for cycle-end as- 
sessment and for determining students' 
SPLs and level advancement. 



ERIC 



aHffiiSie 12: Winter 2000 



22 



ADVENTURES IN ASSESSMENT 



Volume 
12 * 24 




THE WORK CONTINUES 

This curriculum is still a work-in-progress. 
Now it is necessary to critically evaluate it. 
Are the levels more clearly defined? Are 
the expected outcomes for each level realis- 
tic? Do teachers feel that this is a helpful 
guide? How are students responding? 
What else would we like to include? In the 
future, as our program continues to change 



and as expertise grows in the ESOL field, 
there will be more opportunities for teach- 
ers to further develop curricula. Taking on 
this role was a great learning experience 
for me and I have benefited from the 
knowledge I have gained from teachers, 
learners and consultants with whom I have 
worked. 
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/ want someone to come and speak about . 

1 . Ban king 
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2. Healthcare 




3. College or a GED Program 




4. Boston's Public Schools 






5. Unions and Labor Laws l 
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6. Tax Forms 




7. The Library 




8. The Fire Department 




9. The Police Department 




10. Home Mortgages 
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A Performance Framework for Teaching 
and Learning with the Equipped for the 
Future (EFF) Content Standards 



by 

Peggy McGuire 




EFF Assessment 
Coordinator, 
Philadelphia, PA 



E quipped for the Future (EFF) is a 

grassroots and collaborative initiative 
of the National Institute for Literacy, 
aimed at the reform of this country's Adult 
Education and Lifelong Learning System 
so that the latter becomes thoroughly and 
consistently standards-based and cus- 
tomer-driven. Building consensus on the 
results that matter to all the customers of 
the system is the first and most important 
component of this reform since it provides 
the foundation for real change in teaching 
and learning. 

Through research and consensus-build- 
ing over the past several years, we have 
been able to describe the knowledge and 
skills all adults need to be effective in car- 
rying out activities central to their roles as 
parents and family members, citizens and 
community members, and workers. The 
EFF Content Standards express this con- 
sensus on what adults need to know and 
be able to do. And teachers all over the 
country, in ABE and ESOL classes, in Even 
Start and Welfare-to-Work programs, have 
been using the tools of the EFF Content 
Framework — the Role Maps, the Com- 
mon Activities, the Skills Wheel, the Stan- 
dards — to translate learner goals into 
instruction. They are able to do so because 
these components of the Content Frame- 
work reflect a dual focus on building skills 
and applying those skills to achieve real- 
life "results that matter". 

Having established what adults need to 
know and be able to do, our current task is 
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to use a similar research and consensus- 
building process to develop an Assessment 
Framework that supports the Standards. 
This Assessment Framework will allow us 
to measure how well students are able to 
use what they know and report it in a 
meaningful way. It will move us from 
"EFF Content Standards" toward "EFF 
Performance Standards". 

One of the key tasks of this new phase 
of development is to define levels for each 
of the individual EFF standards — specifi- 
cally, to build a research-based perfor- 
mance continuum for each EFF standard 
that will support the identification of level 
descriptors for all 16 standards. 



ADULT COMPETENCE AND A 
"CONTINUUM OF PERFORMANCE" 

To imagine a continuum of performance 
that stayed true to the foundation of EFF, 
we had to first ask what adult "compe- 
tence" looks like. How do we express skill 
development and application as it relates 
to the purposes and role-centered activities 
that adults carry out as they move and 
grow through their lives, adding skills, 
knowledge and abilities that increase their 
flexibility in responding to change? In 
moving from Content Standards describ- 
ing what adults know and can do, to Per- 
formance Standards describing how well 
they know and can do, to which character- 
istics of performance is it important to pay 
attention? 

We began by looking at other frame- 
works that have attempted to define a 
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similarly broad continuum of adult perfor- 
mance, including the National Adult Lit- 
eracy Survey (NALS) and the 
qualifications frameworks developed by 
Australia, England, Scotland, South Africa, 
and New Zealand. We also looked at cog- 
nitive science research on expertise and 
transfer, and data from EFF field develop- 
ment sites that included teacher descrip- 
tions of student performance. Our goal 
was to identify a theory-based set of di- 
mensions for describing performance at 
both ends of the continuum: adults with 
many years of formal education and ad- 
vanced degrees at the high end, and adults 
with few years of formal education and 
low English literacy skills at the beginning. 

We were sensitive to the failure of exist- 
ing adult frameworks to adequately dis- 
criminate among performances at the low 
end of the scale. We paid very close atten- 
tion to data from our field sites that de- 
fined the kind of evidence of progress 
teachers looked for and how they de- 
scribed student performances. At the same 
time, we made the assumption that the 
goal of the adult literacy system for adults 
at the low end of the scale (as for all other 
adults) is to facilitate increasingly more ef- 
fective performance in the world. We 
wanted to be sure to build one continuum, 
and not to strand low-literate adults on a 
special, developmental continuum cut off 
from movement along the main pathway 
toward mastery and expert performance. 

Since research over the past 20 years has 
been building a greater understanding of 
the cognitive and metacognitive strategies 
used by expert performers and how they 
differ from those used by novice perform- 
ers, we began to examine whether we 
could build our continuum on this theo- 
retical foundation. 
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DIMENSIONS Or PERrORMANCE 

We have identified four key dimensions of 
performance to generate detailed descrip- 
tions of learner performance. We will 
eventually place those descriptions on a 
developmental performance continuum for 
each EFF standard. These dimensions are: 

• Depth and breadth of a knowledge base. 

• Degree of fluency and flexibility with 
which one can perform. 

• Degree of independence. 

• Range of conditions under which one 
can perform. 

What is a knowledge base and how do 
we build it? Traditionally we think of a 
knowledge base as "what you know". The 
cognitive science research on expertise and 
transfer asks us to think not only about 
how much one knows (the number of facts, 
procedures, concepts, etc.) but also how 
the knowledge is organized. The goal is to 
assure that as an individual's store of 
knowledge relative to a particular domain 
or skill grows, the structure of the knowl- 
edge base also develops, becoming increas- 
ingly coherent, principled, useful, and goal 
oriented. This means that what a person 
knows - at whatever level of knowledge - 
is organized for efficient retrieval and ap- 
plication in everyday life. She has access to 
that knowledge, and can draw upon it for 
effective action in the world. 

As an individual moves along a devel- 
opmental continuum from "novice" to- 
ward "expert", then she develops more 
and better strategies for organizing the 
contents of her knowledge base around 
principles and concepts. To "organize" is 
to see, and eventually develop new "pat- 
terns" of information. By "patterns" we 
mean connections or relationships between 
1) facts, 2) facts and concepts, and/or 
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Our goal was to 
identify a theory- 
based set of dimen- 
sions for describing 
performance at 
both ends of the 
continuum: adults 
with many years 
of formal education 
and advanced 
degrees at the high 
end, and adults 
with few years of 
formal education 
and low English 
literacy skills at 
the beginning. 
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3) prior knowledge and newly-acquired 
facts, connections or relationships that are 
based on some bigger themes or concepts 
that tie the bits of information together. 

As what an individual knows becomes 
more organized, growing expertise is also 
marked by increased ability to identify 
what information is relevant to a particular 
task or problem. Further, the individual 
becomes more and more able to identify 
the conditions under which particular 
kinds of knowledge are useful. The "ex- 
pert", then, has many strategies to retrieve 
and use information that is appropriate to 
whatever work she is trying to accomplish, 
in whatever context she finds herself. 

We see evidence of such developments 
in the knowledge base in improved perfor- 
mance along the other three dimensions 
we have identified. In other words, a more 
coherent, principled knowledge base sup- 
ports performance with greater fluency, 
greater independence, under a greater 
range of conditions. 

What do we mean by building greater 
fluency and flexibility of performance? We 
are all familiar with the axiom "practice 
makes perfect." EFF defines this dimension 
as the level of effort required for an indi- 
vidual to retrieve and apply relevant 
knowledge. Points along the continuum 
range from "a great deal of effort" through 
"some effort," and "fluent" to "automatic." 

Why performing with increasing inde- 
pendence? An important indicator of an 
adult's increasing skill is the extent to 
which he or she needs direction or guid- 
ance in using that skill. EFF borrows 
DeFabio's definition of independence for 
this dimension: "an individual's ability to 
select, plan, execute, and monitor his or 
her own performance without reliance on 
the direction of others." Points along a skill 
O 
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development continuum for this dimen- 
sion of performance would look at a de- 
creasing need for assistance in carrying out 
these executive (or metacognitive) func- 
tions of performance, whether a person is 
acting alone or in collaboration with oth- 
ers. 

What about increasing the range of per- 
formance? This dimension gets to the heart 
of defining how well an individual can use 
a skill. Included in our concept of "range" 
are variables related to both task and con- 
text. These variables include the type as 
well as the number of tasks and contexts in 
which one can use the skill. Variables to 
consider include the degree of familiarity/ 
unfamiliarity of a task or context; the 
structuredness/ unstructuredness of the 
task; and the complexity of the task. In- 
crease in range, like increase in indepen- 
dence, is directly related to the growth and 
more principled organization of one's 
knowledge base. 

We have focused on these four dimen- 
sions of performance because they address 
not only what people know but also how 
well they can use what they know. To- 
gether, they comprise a simple, coherent, 
research-based picture of performance that 
makes sense within programs as well as to 
all the many publics that care about what 
people can do (and where their limits are) 
as a result of their learning. 

EFF FIELD DEVELOPMENT 

Phase 3 of EFF field development (1999- 
2000) has been engaging practitioners from 
five states (Maine, Ohio, Oregon, Tennes- 
see and Washington) in using these four 
dimensions. Each participating teacher 
from the 15 to 20 field development pro- 
gram sites is currently collecting informa- 
tion about student performance and 
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progress against two EFF Content Stan- 
dards. They are utilizing a new data collec- 
tion template which focuses observation 
and documentation of learner performance 
on the four dimensions of skill perfor- 
mance. Thus the information they provide 
will enable us to construct a performance 
continuum for each standard that is based 
both on practice and on cognitive develop- 
ment. As a result, we will be able to de- 
velop EFF descriptors that correspond to 
the "levels" of other systems (the National 
Reporting System, for instance) and iden- 
tify not only what skills adults have, but 
also what adults can do with those skills. 

The data collection template being used 
for this field research was developed 
through the efforts of a group of teachers, 
EFF staff and Technical Assistance Team 
members who began meeting in Summer, 
1999. We began with a template that ar- 
ticulated the four key dimensions of per- 
formance as a guide for placing these 
descriptions on a developmental perfor- 
mance continuum. 

Teachers found the template too confus- 
ing and suggested that it would be easier 
to use if the dimensions were embedded 
in categories that reflected how teachers 
think about planning and instruction. 
Guided by this insight, we reorganized the 
template to focus on four categories: task, 
context, knowledge, and performance. 

Task and context break out two aspects 
of range of conditions to reflect how teach- 
ers begin planning for appropriate instruc- 
tion. The focus here is what learners can do 
with a particular skill or combination of 
skills. Thus, EFF teachers will likely use 
key components of the EFF Content 
Framework — Role Maps and Common 
Activities — to develop meaningful learn- 
ing opportunities in which knowledge and 
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skills are linked to real-life tasks and con- 
texts. 

Knowledge remains a single dimension. 
Degree of fluency and degree of indepen- 
dence are combined into one performance 
category since teachers found that they 
looked at these two aspects together. All 
three relate to what learners know. Further- 
more, as noted earlier, growing fluency 
and independence relate directly to the ex- 
tent to which knowledge is organized 
around important concepts and principles. 
Thus, a "well-organized" knowledge base 
is the bridge between "knowing" and "do- 
ing". 

To assist teachers in using the four di- 
mensions in planning and instruction, we 
asked questions about each category, as if 
we were developing an "observation ru- 
bric." 

Field development participants are us- 
ing this revised template to develop an on- 
going record of what learners can do with 
specific skills, a collection of evidence "mo- 
ments" over time that together create the 
"big picture" of real-world outcomes. We 
hope the template will help teachers draw 
pictures of learner performance which cap- 
ture the complexity of what learners are 
capable of performing, and communicate it 
in a way that is easy to understand. 

A PERFORMANCE FRAMEWORK 

While the development work described 
here is ongoing, we are well aware that 
many teachers are already using the EFF 
Content Framework and are excited by its 
usefulness in placing learner goals 
squarely at the center of their work. We 
wanted to support their ongoing efforts 
while also providing them with a tool to 
help them to begin to use the four dimen- 
sions to inform their practice. 
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Staying focused on 
the components of 
performance allows 
EFF teachers and 
learners to be sure 
that everyone is 
focused on the 
same important 
things when plan- 
ning, implement- 
ing and assessing 
learning activities. 



To that end, the categories and their re- 
lated performance questions from the data 
collection template have also been orga- 
nized into a one-page document, the "EFF 
Performance Framework" (see Figure 1) 
which is available now for any EFF teacher 
to use in planning, implementing and as- 
sessing instruction around the EFF Content 
Standards. The "EFF Performance Frame- 
work" integrates the four dimensions into 
a set of observation protocols, or questions. 
The questions focus on the most important 
aspects of learner performance for devel- 
oping expertise. These are the questions 
that we hope that every teacher who is 
teaching to the EFF Content Standards will 
ask herself 

• when she is trying to understand what 
her learners already know and are able 
to do; 

• when she is planning instructional ac- 
tivities to fill in gaps and strengthen 
skills; and 

• when she is assessing skill development 
post-instruction, as a result of the 
planned instructional activities. 

This is a research-based framework 
through which to look at learner perfor- 
mance and answer the questions "What do 
students know?" and "What can students 
do with this knowledge?" We hope teach- 
ers will carry this framework around, refer 
to it often, and finally integrate it into their 
thinking about teaching, learning and as- 
sessing with the EFF Standards so well that 
it becomes second nature — that they be- 
come "experts" at focusing their thinking 
on the four dimensions of performance. 
Furthermore, the framework is usable for 
any/ all of the standards, and for collecting 
detailed evidence in a way that it can be 
compared. 
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Using the Performance Framework in 
practice requires teachers and learners to 
stay focused on the components of perfor- 
mance for each standard. A lot of the 
power and promise of the framework 
comes from the fact that the Content Stan- 
dards are consensus statements of what is 
important to teach and learn for each skill. 
The components of performance break 
down what needs to be taught and as- 
sessed to ensure that learners develop and 
can use the EFF skill. Staying focused on 
the components of performance allows 
EFF teachers and learners to be sure that 
everyone is focused on the same important 
things when planning, implementing and 
assessing learning activities. 

In other words, focus on the EFF com- 
ponents of performance allows for greater 
standardization of instruction without sac- 
rificing flexibility, creativity, or accommo- 
dation of diverse learning styles and 
needs. It enables us to reliably say: "Adults 
with this skill can do these things — to act 
flexibly, with a range of options and 
choices, to meet the goals in their lives — 
and this is how we know." 

The Performance Framework, then, is a 
starting point for focusing teaching and 
learning on EFF Standards and on the four 
key dimensions of skill development and 
application. By establishing the four di- 
mensions as the basis for our Performance 
Framework as well as for our developmen- 
tal continuum, EFF aims to help teachers 
and learners keep simultaneous focus 
on EFF skills development (what adults 
know) and on EFF skills application (what 
adults can do with that knowledge). 
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USING THE PERFORMANCE 
FRAMEWORK TO PAY ATTENTION 
TO CRITICAL ASPECTS OF TEACHING 
AND LEARNING 

In order to insure that learners are devel- 
oping the skills that they need to reach 
their goals, the EFF Performance Frame- 
work encourages teachers and learners to 
pay attention to the following aspects of 
performance before, during and after in- 
structional activities focused on the EFF 
Content Standards: 

1) Tasks 

(Dimension: range of conditions under which 
one can perform) 

What kinds of tasks can learners per- 
form? 

What are the activities that learners en- 
gage in that require use of the skill? These 
tasks might be carried out in the instruc- 
tional setting and might be contained in a 
discrete lesson or a series of lessons. Or, 
learners may be performing the tasks in 
other settings, such as at home or in their 
communities. In either case, the task will 
be identified because it requires use of the 
targeted EFF skill as well as relates to a 
learner's goal or purpose. 

How complex is the task? Is it simple, 
one-step, brief, short-term, well-defined 
and highly structured, requiring very little 
judgment or prediction of outcomes? Or is 
it more complex, longer and more sus- 
tained, involving multiple steps, less de- 
fined or structured, or even self-defined 
and self-initiated, requiring careful judg- 
ment and accurate prediction of outcomes? 

How familiar is the task to the learner? 
Some guiding questions would be "Has 
the learner done this task before? Has the 
learner seen this task done before?" 
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EXAMPLE: 

You are focusing on the EFF Standard 
"Use Information and Communications 
Technology", and your learners' goals in- 
volve using computers well enough to pro- 
duce a publication that can communicate 
important information to others about a lo- 
cal community organizing effort. Reaching 
this goal will require mastery of a large 
range of tasks. 

Complexity: A simple task might be 
safely turning on and off the computer, or 
somewhat more complex, safely entering 
and exiting a word processing program. 

As the learners' skills develop, they will 
engage in ever more complex tasks, from 
writing and saving short written docu- 
ments, to editing and rearranging text; 
highly complex tasks, requiring strong 
skills, might include using graphics, mul- 
tiple fonts, even a desktop publishing pro- 
gram to write and produce a professional 
newsletter. 

Familiarity: Has the learner ever seen a 
computer before? Ever used a computer? 
Ever used a word processing program? 
Ever worked with publishing software? At 
the high end of the continuum, the learner 
has performed the task at hand many 
times before, or has seen it done often be- 
fore; thus it is a familiar task. Further 
down the continuum, tasks are less and 
less familiar to the learner — all the way to 
those that the learner has never before per- 
formed or seen done. 

2) Contexts 

(Dimension: range of conditions under which 
one can perform) 

In what contexts can learners perform? 

How familiar are the learners with the 
context? 

Carrying out a task in a very familiar 
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Movement along 
[a] continuum is 
marked by ability 
to perform [a] 
task in a growing 
number of 
different contexts, 
indicating that 
the learner is 
transferring the 
use of EFF skills 
from one activity 
to another. 



situation where there is a lot of assistance 
and support is easier for learners than car- 
rying out the same task in a less familiar, 
less supported context. At the high end of 
the continuum, the context is very familiar 
— for instance, exclusively in the class- 
room or instructional situation. Further 
down the continuum, the context becomes 
less familiar — on the job, in a community 
meeting, in the home where the learner has 
never tried to accomplish this particular 
task before. 

In how many different situations can the 
learners perform? 

At the low end of the continuum, the 
learner can only accomplish the task in a 
single situation (such as the instructional 
setting). Movement along the continuum is 
marked by ability to perform the task in a 
growing number of different contexts — 
indicating also that the learner is transfer- 
ring the use of EFF skills from one activity 
or role to another. At the highest point, the 
situations in which the learner can perform 
are multiple and varied; learners can per- 
form the task to meet a variety of needs 
and purposes, and their transfer of skill 
use from one context to another is system- 
atic. 

How much risk is involved in the situa- 
tion? how high are the stakes? 

The external environment in which 
learners are trying to accomplish goals can 
often present significant challenges to 
achievement. Some of these challenges 
have to do with "why" the learner is per- 
forming a task. What is at stake? Is this 
about successfully following directions in 
class? passing a credentialing exam, like 
the GED? getting /keeping a good job? An- 
other aspect of "risk" has to do with more 
personal and/ or societal challenges that 
learners often face in their efforts. Is there a 
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threat of abuse in the home? Are there is- 
sues of racism/ sexism/homophobia/etc. 
in the community? Are decent jobs un- 
available, or in settings that pose risks to 
workers? 

At one end of the continuum, such risks 
are minimal, and the stakes are low. At the 
other, risks /stakes increase. 

EXAMPLE: 

You are working with the EFF Standard 
"Speak So Others Can Understand", and 
learners are hoping to present testimony at 
upcoming statewide hearings on Adult 
Education funding. 

Familiarity: At the low end of the con- 
tinuum, the context for speaking in front of 
a group will be most familiar to the learn- 
ers — in the learners' own instructional 
setting, in front of a small group of fellow 
learners, perhaps. Further along, the con- 
text becomes less familiar — in a different 
classroom, in front of a different group of 
learners, or the agency staff, or its Board. 
The context of the state hearings would be 
high on the continuum, assuming learners 
have never spoken in front of state legisla- 
tors and so are unfamiliar with that con- 
text. 

Number of different situations: At the low 
end, learners can speak before a group 
only in one situation — the instructional 
setting in this example. But as develop- 
ment along the continuum proceeds, learn- 
ers can perform outside the instructional 
setting and in increasing numbers of situa- 
tions — in other classes, at Staff or Board 
meetings, before the Home-School Asso- 
ciation, at state hearings. 

Level of risk: Relatively low-risk/low- 
stakes contexts, on the low end of the con- 
tinuum, might include a classroom full of 
supportive co-leamers or before other au- 
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diences who will not make major decisions 
about the learner based on the perfor- 
mance. Even at the state hearings, where 
learner performance may have a profound 
impact on listeners, the risk to the indi- 
vidual learners is not at the highest end of 
the continuum. But learners may soon 
need to use their speaking skills in much 
higher-stakes contexts — before potential 
employers or in an attempt to gain a seat 
on the local school board, for instance. 

We also need to understand, if possible, 
the personal challenges that contribute to 
the level of risk. For instance, one impact 
of the psychological effects of various 
forms of abuse can be eroded self-confi- 
dence which makes speaking in any public 
situation a very high-risk activity. Then, 
what may otherwise seem a low-risk con- 
text needs to be accounted for at a higher 
point on the continuum. 

3) Knowledge Base 

(Dimension: depth and breadth of knowledge 

base) 

When we consider the Knowledge Base 
necessary for any EFF skill, remember that 
we are looking at skills in the contexts of 
purpose (what adults need to know to 
meet their expressed goal) and perfor- 
mance (what adults need to be able to do 
with what they know). So we need to look 
at content knowledge, but we also need to 
look beyond content knowledge to how we 
organize and apply the content knowledge 
in meaningful contexts. This focus on use 
of skills to meet specific purposes may ex- 
plain why so much "EFF Teaching" so of- 
ten looks like "authentic task" or "project" 
based learning (content + application) in- 
stead of "drill and practice" (content only). 

The following two questions refer to 
what is contained in the knowledge base 
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for the targeted EFF skill; the third ques- 
tion, and its subset of questions, refer to 
specific strategies for organizing and ap- 
plying the contents of Knowledge Base for 
use in a meaningful context. 

What do learners know? 

What vocabulary do the learners have 
related to the skill? related to the subject 
area? 

Depending on the task to be accom- 
plished, learners will need to understand 
different amounts and kinds of language 
used in the subject matter, as well as lan- 
guage about the skills being developed in 
the task. At the low end of the continuum, 
learners will only have minimal and 
simple vocabulary; as they move along in 
development, their store of vocabulary will 
grow and begin to include less familiar 
and more technical terminology. 

What content knowledge do the learners 
have related to the skill? related to the sub- 
ject area? 

"Content knowledge" for any skill or 
subject area includes familiarity with facts, 
operations, concepts, rules, protocols, prac- 
tices and/ or conditions of use essential to 
the skill/ subject area — including the pur- 
pose and audience for skill performance. 

At the low end of the continuum, learners 
have minimal or no familiarity with these 
essential content aspects of the skill/sub- 
ject. As learners develop the skill, they be- 
come more and more familiar with an 
increasing amount of content knowledge 
that is useful in a greater variety of tasks 
and for a greater variety of purposes. 

What strategies do the learners have for 
organizing and applying content knowl- 
edge? 

Before we can use what we know about 
a skill, we have to have ways to organize 
all those discrete bits of skill-related infor- 
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As learners develop 
more strategies, 
their understand- 
ing increases and 
becomes more com- 
plex; they can 
regularly recognize 
patterns, some 
simple and some 
“ higher order. " 
They can recognize 
cause and effect 
relationships, for 
instance, and can 
join prior knowl- 
edge with new in- 
formation to solve 
some problems. 

They begin to 
generalize, draw 
conclusions and 
predict outcomes 
in some cases. 



mation that come into our brains in differ- 
ent ways and at different times. That way 
the information "bits" can eventually be 
retrieved in the right combinations or 
"chunks", at the right time, for the right 
purpose. 

Can learners recognize relationships or 
connections? 

Can learners create new relationships or 
connections? 

These questions refer to strategies that 
overlap in the developmental process so 
we deal with them together. The strategies 
have to do with the growing ability to first 
see, then develop, new "patterns" of infor- 
mation. By "patterns" we mean connec- 
tions or relationships between facts, facts 
and concepts, and/ or prior knowledge and 
newly acquired facts, connections or rela- 
tionships that are based on some bigger 
themes or concepts that tie the bits of infor- 
mation together. 

At the low end of the continuum learn- 
ers have very few such strategies and are 
limited to simple recall of previously- 
learned bits of information. Initial "pattern 
recognition" is evident later when learners 
can achieve some low-level understanding 
of meaning by explaining, interpreting, 
translating, summarizing, paraphrasing, 
restating, and/ or using examples. As 
learners develop more strategies, their un- 
derstanding increases and becomes more 
complex; they can regularly recognize pat- 
terns, some simple and some higher order. 
They can recognize cause and effect rela- 
tionships, for instance, and can join prior 
knowledge with new information to solve 
some problems. They begin to generalize, 
draw conclusions and predict outcomes in 
some cases. 

As they move toward the upper reaches 
of the continuum, learners begin to move 
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beyond "pattern recognition" to "pattern 
creation." They move from accurate analy- 
sis (seeing the relationships between con- 
cepts and related details, between content 
and organizational structure — using strat- 
egies such as comparison/contrast, analo- 
gies, generalization, inference and 
prediction) to synthesis (organizing infor- 
mation in new ways and proposing alter- 
nate systems of knowledge — using 
high-order strategies such as abstraction, 
criticism and justification). 

Can learners identify what information 
is important to the task/problem? 

At the low end of the continuum, learn- 
ers who do not have data/information or- 
ganized around principles and concepts 
have a very difficult time deciding what 
information is relevant to solving a prob- 
lem or completing a task. Further along, 
the strategies learners have for organizing 
information enable them to consciously re- 
trieve important information for a clearly- 
defined purpose, then for multiple 
purposes, then for wide-ranging purposes 
and contexts. 

Can learners understand when informa- 
tion or concepts apply? 

This question refers to a learner's ability 
to decide which procedures, concepts or 
principles are applicable to which situa- 
tion/task/ problem — in other words, the 
conditions under which procedures or con- 
cepts are useful. At the low end of the con- 
tinuum there is very little understanding 
that procedures or concepts are not univer- 
sally applicable. Further along, learners 
develop a growing repertoire of strategies 
linked to specific situations. Eventually, 
learners are able to flexibly choose from 
among a range of appropriate strategies 
those that are most effective under the spe- 
cific combination of circumstances repre- 
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sented by task and context. 

EXAMPLE: 

Your learners are mostly single moms 
involved in a "Welfare-to-Work" program; 
they are developing the EFF Skill "Convey 
Ideas in Writing" and are engaging in a se- 
ries of tasks to do so in contexts that are 
meaningful to them. 

Vocabulary and content knowledge: Per- 
haps the learners need to begin with rela- 
tively simple tasks, like making shopping 
lists before going to the grocery store so 
that they will use their limited funds more 
carefully. This task would appear at the 
lower end of the continuum as it requires a 
limited set of vocabulary words, basic con- 
tent knowledge, and limited familiarity 
with writing rules and practices. Further 
along the continuum, tasks will require a 
larger store of vocabulary, more diverse 
content knowledge and broader familiarity 
with writing rules — a note to a child's 
teacher requesting a meeting, for instance. 
While this may not be highly complex, it 
nevertheless involves sentences, punctua- 
tion, coherence, etc. as well as attention to 
the needs of an external audience to whom 
you are conveying information. Still fur- 
ther on, a written description of one local 
child care program may require a good 
store of words, writing conventions, and 
knowledge of child care concerns. At the 
high end, a task such as writing a guide to 
local child care options would probably re- 
quire use of previously unfamiliar and 
more technical vocabulary, considerable 
mastery of writing conventions, and sig- 
nificant content knowledge in the areas of 
child development, state licensing regula- 
tions, etc. 

Relationships and connections: At the low 
end of the continuum, learners may be able 
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to do no more than copy in writing the 
words that they need for a grocery list 
from a master list (or later, from a newspa- 
per circular). A little further on, they may 
be able to write a simple note to a teacher 
after practicing doing so in class, but this 
activity still represents little "understand- 
ing" beyond recall. However, a task such 
as rewriting the contents of a brochure ad- 
vertising a local child care program so that 
it is easier to read requires at least "low- 
level" understanding. That understanding 
is reflected in the ability to interpret, para- 
phrase and restate information in writing, 
and so belongs further along the con- 
tinuum. As learners progress in skill devel- 
opment, they are able to see patterns and 
use strategies to express relationships. For 
instance, after rewriting several brochures, 
they are ready to write an essay in which 
they compare the programs that have pro- 
duced the brochures — what do the pro- 
grams have in common? how are they 
different? And high on the continuum, 
learners will be able not only to recognize 
patterns but to create new ones. To write a 
useful guide to child care options, their 
writing skills will allow them to summa- 
rize their own research and prior knowl- 
edge, critically analyze choices according 
to key points of interest to parents, and ex- 
press their conclusions and recommenda- 
tions as to the best options. 

What information is important, and 
when? 

At the low end of the developmental 
continuum, a learner might copy exactly a 
master grocery list or a "generic" note to a 
teacher, without realizing that the list can 
be tailored to individual needs at specific 
times, or that a note will be more effective 
if it shows awareness of a particular 
teacher's context and concerns. Even re- 



As learners 
progress in skill 
development, 
they are able to see 
patterns and use 
strategies to 
express 
relationships. 
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"Performance" 
refers to the dimen- 
sions of skill devel- 
opment that move 
the focus of teach- 
ing and learning 
beyond "what we 
know" and "how 
we organize what 
we know" to "what 
we can do with 
what we know". 



writing a brochure loses some impact if the 
rewrite doesn't edit out information that is 
not useful for the audience's purpose (e.g., 
trying to make a decision about which 
child care option to choose). As skill devel- 
ops, the learner becomes more and more 
able to choose and communicate informa- 
tion based on key points of interest (the 
"right" foods for my new, healthy diet; 
two specific matters I want to discuss 
about my child's reading; adherence to 
state child care regulations) and the par- 
ticular audience being addressed (myself; 
the Reading Resource Room teacher; single 
moms in search of quality and affordable 
child care). 

4) Performance 

(Dimensions: degree of independence; degree of 
fluency and flexibility with which one can per- 
form) 

How well can learners perform? 

"Performance" refers to the dimensions 
of skill development that move the focus of 
teaching and learning beyond "what we 
know" and "how we organize what we 
know" to ''what we can do with what we 
know". It is about what the use of an EFF 
skill looks like in practice. 

How fluently can learners perform? 

How much effort is required? 

In the early stage of skill development, 
learners have a difficult time using the 
skill — partly because of lack of knowl- 
edge, and partly because of lack of prac- 
tice. To perform at all, learners constantly 
need to tap what we call "working 
memory" and therefore have little energy 
left to absorb and understand the informa- 
tion available to them. So, at the low end of 
the continuum, performance of an EFF skill 
will look slow, difficult, and requiring 
great effort on the part of the learner. As 
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knowledge and experience grow, learners 
will be able to use the skill to perform 
some tasks with greater ease, while other 
more complex or unfamiliar tasks will still 
require noticeable effort. At the high end 
of the continuum, learners can use the skill 
effortlessly, "automatically," with a high 
degree of fluency , to do whatever they 
need to do. Performance that used to re- 
quire working memory now seems "un- 
conscious". 

How consistently do the learners start 
and finish, getting to the desired outcome? 

The slowness and difficulty in perfor- 
mance at the low end of the continuum is 
matched by inconsistency of performance. 
At this point, learners will make a lot of 
"errors", will produce little work, and will 
have a hard time finishing the task. Fur- 
ther along the continuum, learners begin to 
show greater consistency in use of the skill; 
they complete tasks more often and with 
fewer errors. At the high end, effective 
skill use is systematic, work is completed 
and errors are rare. 

How well are barriers controlled or 
overcome? 

"Barriers" here refer to immediate 
adverse conditions that get in the way of 
effectively using a skill to perform a task. 
Such barriers may differ in nature and de- 
gree depending on the task and context (is 
the room too noisy? do I need glasses? do 
I never find time to work at home? have I 
misplaced the instructions?), but the key 
question is if/how the learners act to ad- 
dress them. At the low end of the con- 
tinuum, learners will be easily diverted 
from the task by such problems, will be de- 
feated and give up. Further along in skill 
development, learners will start to more 
often strategize about how to overcome 
identified obstacles; at the high end, regu- 
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larly addressing and overcoming barriers 
becomes part of the learning process 

EXAMPLE: 

You are working on development of the 
EFF Skill "Use Mathematics in Problem- 
Solving and Communication". Your learn- 
ers have decided that they want to plant a 
community vegetable garden so that they 
can learn work skills by starting a small 
business selling the vegetables. The range 
of increasingly complex math tasks in- 
volved in planning a garden may include 
basic computations, simple measurements, 
areas, perimeters, even drawing and using 
scaled blueprints. 

At the early stages on the continuum, 
learners will need to be taught, and con- 
stantly reminded of, the math operations 
that they need to complete necessary plan- 
ning activities (how much space do we 
have? what shape will the garden take? 
how many different plots can we put in? 
how many plants can be placed in each 
plot? etc.). They will work slowly and 
struggle to "get it right". They may have a 
difficult time with obstacles that get in 
their way (do we have the right measuring 
tools? how do we get them? One plant re- 
quires more space than another, so how do 
we plant them both in the same plot?). It 
may be easy to abandon their plans at this 
point if too many problems present them- 
selves. But as skills and experience with 
the math increase, learners will need fewer 
reminders and less help with addressing 
barriers. At the high end of the continuum, 
their work is systematic and they get the 
garden planted according to a clear and 
successful plan — using math to achieve 
their goal. 

How independently can the learners 
perform? 



How much help is needed from others? 

When we use "Independence" as a di- 
mension of skill development, we are not 
suggesting that working alone is better or 
"smarter" than working in collaboration 
with others. However, one indicator of de- 
veloping mastery has to do with how 
much assistance learners need in order to 
use a skill to perform a task. At the low 
end of the continuum, learners need sub- 
stantial help from others in order to use the 
skill even in the most familiar and simple 
tasks. Then, as the skill develops, learners 
will still need help, but more often with 
tasks that are difficult or unfamiliar. At the 
high end, no assistance is needed; rather, 
learners at this level are ready to assist oth- 
ers. 

How much initiative is shown in getting 
started? 

At the lower end of the continuum, 
learners need a "push" to begin a task; as 
they develop greater skill, they will need 
less prompting and will often get started 
on their own. At the high end learners 
need no "push" to get started; in fact, they 
will often initiate new tasks on their own, 
identifying and pursuing new opportuni- 
ties to learn. 

How often do learners generate their 
own strategies to complete the task? 

Once learners get started, how much 
and what kinds of help do they need to 
complete a task? At the low end of the con- 
tinuum, learners need to be offered a great 
deal of structure, clarification and guid- 
ance. They need to be able to "copy" strate- 
gies and approaches that others have used. 
Further along the continuum, learners can 
sometimes come up with strategies on 
their own, without strong guidance; at 
other times they still need approaches to be 
imposed and guidance to be offered. At the 
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When we use 
"Independence" as 
a dimension of skill 
development, we 
are not suggesting 
that working alone 
is better or 
"smarter" than 
working in collabo- 
ration with others. 
However, one indi- 
cator of developing 
mastery has to do 
with how much 
assistance learners 
need in order to use 
a skill to perform a 
task. 
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In the classroom, 
teachers and learn- 
ers have discovered 
the power of the 
EFF Content 
Framework to align 
skill development 
with learner goals, 
and to focus in- 
struction both on 
what learners know 
and on what they 
can do with that 
knowledge. 



high end, learners can invent their own 
strategies, adapt approaches from outside 
sources of information, and justify their 
choices of the most appropriate ways to 
complete the task. They don't need guid- 
ance but can offer guidance to others. 

EXAMPLE: 

You have heard several individual 
learners, at various times, complain about 
conditions at your agency ("I wish we had 
some space here to just hang out and re- 
lax"; "we need a lunch room"; "I'd like to 
have books that we can take home to 
read"). You want to find a way to turn 
"complaints" into constructive input, so 
you decide to focus on the EFF Standard 
"Advocate and Influence". You approach 
some of the students and introduce the 
idea of forming a Learner Leadership 
Group, and they agree. 

At the low end of the continuum, learn- 
ers have little or no experience with effec- 
tive advocacy, so they will need a great 
deal of instruction, prompting and assis- 
tance from you in how to begin. They will 
need you to give them a highly structured 
strategy, for instance, for recruiting other 
learners to join them and for identifying 
the issues that they want to address as a 
group. They will need your active involve- 
ment in order to insure success at reaching 
even a relatively simple goal such as ask- 
ing Board members of the agency to do- 
nate used books for a small lending library. 
As their skill development moves along 
the continuum (and as they gain experi- 
ence — and success!), they will need less 
assistance and fewer instructions from 
you, though they may still want your help 
with more complex advocacy opportuni- 
ties (getting permission to reconfigure cur- 
rent space, or raising funds to add on a 
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new room!). At the high end of the con- 
tinuum, learners will be so skilled in advo- 
cacy, and will need so little help, that they 
can assist other, new program participants 
in joining the group and learning the nec- 
essary skills. 



CONCLUSION 

The past five years have been an exciting 
time of national collaboration and innova- 
tion for the EFF initiative. Through a broad 
consensus-building process we have devel- 
oped standards that accurately reflect what 
adults need to know and be able to do. 
Through an iterative field review process 
we have made sure that our standards fo- 
cus on performance that is observable and 
measurable, and that they are specific 
enough to guide instruction and assess- 
ment. In the classroom, teachers and learn- 
ers have discovered the power of the EFF 
Content Framework to align skill develop- 
ment with learner goals, and to focus in- 
struction both on what learners know and 
on what they can do with that knowledge. 

Our remaining tasks are to define mul- 
tiple levels of performance for which the 
students should strive, and to develop an 
assessment framework for EFF to help us 
identify and develop accurate assessment 
tools to meet a range of assessment pur- 
poses. These tasks are critical next steps if 
our standards are not only to guide teach- 
ing and learning but also to frame account- 
ability for results - for learners who need 
to know they have credentials that convey 
what they know and can do to the outside 
world, and for funders and policymakers 
who need to know that programs and sys- 
tems are achieving desired results. 

We have tried here to describe some of 
the key work that is under way in this next 
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phase of the EFF development process: 

• imagining one, research-based Perfor- 
mance Continuum that stays true to the 
principles at the foundation of the EFF 
Content Standards, and that frames for 
teachers the "scope and sequence" of 
adult development as a movement of 
ALL adults — including our learners — 
from "novice" to "expert"; 

• engaging in a field development process 
aimed at building such a Performance 
Continuum for each of the 16 EFF Stan- 
dards that will support identification of 



level descriptors representing real world 
benchmarks; 

• offering a Framework that teachers who 
are using EFF can use now to focus as- 
sessment, planning and instruction on 
four theory-and-practice-based dimen- 
sions of adult skill performance. 

We are deeply grateful to the many de- 
velopment partners whose efforts have 
brought us this far and are moving us for- 
ward. Hard work lies ahead, but we look 
forward to the opportunities and potential 
for real change that accompany it. 



For more information on Equipped for the Future and its publications , contact 1-877-433-7627 or 
go to its website @ http://www.nifl. gov/ejf. 
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Connecting the ESOL Framework 
to Actual Practice 



Rosann was part of a statewide curriculum 
frameworks project in Massachusetts. Funded 
by the Department of Education and facilitated 
by the School for International Training in 
Vermont, this project involved connecting the 
Massachusetts Adult ESOL frameworks to 
practice. This is the account of Roseann's expe- 
rience as part of that project. This article is 
adapted from the draft document "Engaging 
Learners and Practitioners with the Adult 
ESOL Frameworks." June 1999. Roseann 
worked at the Lawrence Adult Learning Center 
during this project. 



by 

Roseann Ritter 



North Andover, MA 



T here were about 35 students in my 
two ESOL dasses. We met five days 
a week for two hours each morning. 
My students represented a cross-section of 
learners ranging in age from 22 to 64 with 
diverse educational backgrounds. The ma- 
jority of students were Hispanic with 2 
Russian students, two Haitian students, 
one Chinese student, one Cambodian stu- 
dent and one Korean student. The classes 
were somewhat homogeneously grouped 
as a high beginner-low intermediate level. 
They were all highly motivated to learn 
the language, however, as adults trying to 
juggle family, work, school, and the many 
pressures involved in being immigrants. 
They had many obstacles to overcome. 

The environment of our class was par- 
tidpatory, with the primary emphasis on 
oral communication and navigating sys- 
tems. Some daily focus was also on read- 
ing and written communication. Oral 
communication was encouraged and fos- 
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tered by guiding students into peer con- 
versations and guided dialogues. There 
was a special emphasis on problem solv- 
ing. My students responded particularly 
well when the dialogues had special mean- 
ing and relevancy to their lives. Because of 
the participatory nature of my classes, the 
learner-input project was very enjoyable 
and worthwhile. 

The primary purpose of the learner in- 
put project was to investigate ways in 
which teachers could develop instructional 
materials and comprehensive units of 
study that would be relevant, meaningful 
and responsive to students' needs. The ul- 
timate goal was to tailor our program to 
better provide our students with the lan- 
guage and survival skills necessary to 
function effectively and meet the chal- 
lenges of everyday life in this country. As 
part of the DOE Learner Input Project, my 
specific purpose was to see if the expressed 
needs and goals of my students matched 
with those expressed in the Massachusetts 
ESOL Curriculum Frameworks. 

My project involved eliciting student's 
feelings and ideas regarding what is im- 
portant to them and what kinds of knowl- 
edge and skills they hope to gain from 
our program/my classroom. We did this 
in the form of a student-generated needs 
assessment. 

Our project evolved into several stages. 
The initial stage involved the learners 
identifying situations in their everyday 
lives that presented the greatest problems 
and challenges to them in trying to com- 
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municate in English. The process began as 
a class discussion where students shared 
their experiences, their concerns, and prob- 
lems that they had encountered. Several 
students willingly shared stories where 
their inability to communicate in English 
presented problems that left them frus- 
trated and embarrassed. As they shared 
their stories, a common bond was estab- 
lished as they realized that they all had 
similar feelings of frustration and despair. 
As the students talked about the contexts 
where they had the most difficulty com- 
municating, I wrote the situations on an 
easel pad. 

After coming up with nine areas of con- 
cern, I passed out index cards numbered 
one to four and asked them to prioritize 
them by listing the most difficult situation 
first, the next second and so on. The results 
were: 

1. Doctors and Illness 

2. Communicating at Work 

3. Police and Court 

4. Talking on the Telephone 

5. Credit 

6. Children's School 

7. Computers 

8. Shopping 

9. Banking 

The following day the class separated 
into focus groups according to the context 
category that they had listed as their first 
priority. Those that felt they had problems 
with "Doctors and Illness" made up one 
group; "Police and Court" made up an- 
other, "Talking on the Telephone" and 
"Communicating at Work" another. Their 
task was to come up with specific prob- 
lems that they had encountered in each 
O 
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category. These were examples from each 
context: "I can't explain my health problem 
to the doctor," "I have trouble when I have 
to go for a job interview," "I can't explain 
my problems to a lawyer," "When my boss 
explains my job responsibilities, sometimes 
I don't understand him and then I get into 
trouble." 

Each team appointed a secretary to 
write down the group's ideas. As I circu- 
lated among the groups, I assisted where 
necessary to help clarify ideas. It was 
stressed that at this particular time the 
main goal was to elicit ideas; grammar and 
spelling were not issues to worry about. 
Students were allowed to express and 
write their ideas in their native language if 
that was the way they could express them 
most succinctly and if all members of the 
group shared a common native language. 
Other members of the group helped to 
translate so the secretary could write it in 
English. The grammar skill that we were 
learning in class at that particular time was 
'superlatives'. Students were able to apply 
the skills that they had learned e.g., most 
difficult, hardest, most confusing, most 
embarrassing, worst etc. in a contextual 
way. Integrating grammar into these dis- 
cussions was important because it gave 
them practice using grammar structures 
we covered in class together by practicing 
oral language. Many students needed to 
know that they were getting the "gram- 
mar" for which they always craved. 

The groups were very engaged in the 
task. At the onset of the project, I explained 
to them that they were playing an impor- 
tant role in assisting teachers to plan les- 
sons and identify content areas that were 
important and meaningful to students 
across the state and that there were other 
adult students engaged in the same pro- 
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Students shared 
many interesting 
stories that helped 
to give me further 
insight into the 
needs of my 
language learners. 



cess. They took complete ownership of the 
project and expressed several times that 
they felt it was a very important endeavor. 

When the groups completed their lists of 
"can't do"s, they dictated them to me and I 
wrote them on sheets of easel paper. I then 
asked, "Does everyone have difficulty with 
all of these things?" From that discussion 
we came up with varying degrees of diffi- 
culty. "That's easy for me, sometimes I can 
do that, it's a big problem," etc. 

The next phase was to convert those 
"can't do"s to "can do"s. This was a diffi- 
cult but necessary step to facilitate dealing 
with the students' varying degrees of diffi- 
culty with the items they came up with. It 
was also putting their thoughts in a "posi- 
tive" rather than "negative" context, em- 
phasizing not only what they " don't know" 
and " can't do" but also what they "do 
know and "can do". Statements like, 

"When my boss gives me new job respon- 
sibilities, I don't understand what he is 
saying and I sometimes get into trouble," 
were changed to, "I understand what I 
need to do when my boss gives me a new 
job responsibility." 

The students again worked in their fo- 
cus groups to convert their can't do's. Each 
group came up with a new list of can do 
statements that they dictated to me and I 
wrote them on the easel pad. Their enthu- 
siasm was truly exciting and their produc- 
tivity was amazing! They were totally 
engaged in every step of the process. After 
our lists were compiled, we were then 
ready to transfer our work to produce a 
needs assessment chart (see pages???) 

Part of our program was to integrate 
technology into what we were teaching in 
class. This was a perfect opportunity to do 
this. Since learning computer skills was 
very important to most students, they took 
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this on with enthusiasm. 

This was the last step in finalizing our 
needs assessment tool. Using the computer 
in my classroom I taught students how to 
produce a table using Microsoft Word and 
how to insert symbols into a document. 
Students were eager to go to the computer 
lab the following week to produce their 
tables. Every student in both classes pro- 
duced a table, inserted their "can do" list, 
and very ingeniously selected symbols to 
insert in their charts. Four needs assess- 
ment surveys were developed based on 
what students felt were the four highest 
priority areas: going to the doctor, police 
and court, talking on the telephone, and 
workplace issues. 

We ran off copies of each of the four 
completed surveys to be administered to 
most of the day and some of the night 
ESOL students in our center. Two of my 
students expressed willingness to adminis- 
ter the survey to the lowest levels where 
some translation would be necessary. They 
accompanied me when I administered the 
survey and translated the items from En- 
glish to Spanish. One of my Russian stu- 
dents translated from English to Russian. 
Another student took the results home and 
tallied every item to produce final figures 
for the survey. We surveyed about 100 stu- 
dents in all four categories. 

Students shared many interesting stories 
that helped to give me further insight into 
the needs of my language learners. Al- 
though many of the needs they expressed 
are the usual concerns that students have, 
there were some new ideas that surfaced 
through discussion. One of the areas that 
generated the most lengthy and interesting 
discussions was the area of law enforce- 
ment and our legal system. Students' con- 
cerns about these systems really got me 
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thinking about what my role is, or should 
be in assisting them to navigate the legal 
system. It became apparent to me that if 
this issue concerns my students to the de- 
gree that they expressed in our discussion 
groups, it is an area that needs to be fur- 
ther explored. Is it our responsibility to 
educate our adult students about the basic 
laws of this country? Should we teach 
strategies that will assist our students in 
advocating for themselves within the law 
enforcement system? Where do you draw 
the line in encouraging students to take 
risks and advocate for themselves? 

This Learner Input Project was a valu- 
able experience for my students and me. 

It enabled me to take a closer look at my 
teaching, and my role as a teacher and to 



formulate some new ideas for future planning. 

As a result of this project, I am currently 
facilitating a team of teachers working on 
developing comprehensive units based on 
the results of the survey's findings. Using 
the needs expressed in the student-gener- 
ated survey we will develop units of study 
that will be closely tied to the Curriculum 
Frameworks and will address the ex- 
pressed needs of our students. We have 
developed several lesson plans on "Doc- 
tors and Illness." We will then begin a sec- 
ond unit on 'Police and Courts." 

Hopefully, this will continue to be an on- 
going process where increasing numbers 
of learners and practitioners will partici- 
pate in planning meaningful and relevant 
curriculum for our center. 



BT 
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impossible. 
I can’t do 
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No way! 
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difficult for 
me to do. 
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Difficult for 
me. 

Sometimes I 
Can do this. 
























I can do 
this most of 
the time. 
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I can do 
this; 

No problem. 
























DOCTORS AND ILLNESS 
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1 . I can explain my health problems to the doctor. 


2. When my child is sick, I can tell the doctor about my 
child’s problem 


3 I can answer all of the doctor’s questions. 


4. .1 can understand when the doctor speaks to me about 
my health. 


5. I can name every part of my body. 


6. I can understand medical language. 


7. I can fill out medical information forms at the doctor’s 
office and at the hospital. 


8. I can go to the pharmacy to get a prescription. 


9. I can read and understand directions on the prescription 
container. 


10. I can read a thermometer. 


11. I can understand what “normal” temperature is. 
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this; 
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12. I can understand what “normal” is for blood pressure 
and cholesterol. 


13. I can make an appointment to see the doctor. 


14. I can telephone for an ambulance. 


15. I can drive to the nearest hospital. 


16. I understand when to go to the doctor and when to go to 
the hospital. 


17. I understand where the special medical clinics are in my 
city. 
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It’s impossible. 
I can’t do this. 






















This is very 
difficult for me to 
do. 






















This is difficult for 
me. Sometimes I 
can do this. 






















I can do this 
most of the 
time. 






















I can do this; 
no problem. 






















TELEPHONE 


I can understand when someone speaks to me 
in English on the telephone. 


I feel comfortable answering the telephone. 


I can respond in English when someone calls 
me on the phone. 


I can look up numbers in the telephone book. 


I can use the yellow pages. 


I can call the child-care center and speak 
English to the caretakers. 


I can call in sick at the Adult Learning 
Center. 


I can call into work and explain why I can’t 
come to work. 


I can call my doctor’s office to make an 
appointment. 


I can call 911 in an emergency. 
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I can do 
this most of 
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I can do this no 
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TELEPHONE 


I can make long distance 
phone calls 


I can call information to 
find out a number 


I can read and understand 
all the charges on my 
telephone bill 


I can call 91 1 in an 
emergency 
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Difficult for 
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Sometimes I 
can do this. 
























I can do 
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I can do 
this; 

No problem. 
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1 . I can fill out a job application. 


2. I can write a resume. 


3. I can interview for a job. 


4. I can communicate with my American co-workers. 


5. I understand my job responsibilities. 


6. I can explain my job to others. 


7. I feel comfortable when my boss gives me a new job to 
do. 


8. I can read directions and work orders. 


9. I can speak to customers and I can understand when they 
speak to me. 


10. I can speak English well enough to get a promotion. 


11. I can teach my job to a new employee. 
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To TABE or Not to TABE: 
One Agency's Options 



by 

Bemie Driscoll 




Taunton Adult 
Literacy Center 
Taunton, MA 



T he Taunton Adult Literacy Center 
offers several programs. Our classes 
consist of two ABE, two ESOL, a 
Pre-GED class and a GED class. We hold 
classes two or three times a week from 9-12 
in the morning. Our student ratio is 60% 
women and 40% men. Some of the stu- 
dents are employed and others are not. 
Many of the students work in the home. 
The ages of the students range from 18-65. 

The goal of our program is to meet the 
needs of the individual student. People 
participate in our program for various 
reasons. Some come to improve their lan- 
guage skills; others seek better employ- 
ment or to continue their education. We 
try to help students achieve their personal 
goals. 

As the assessment person, I talk with the 
student at intake and find out his or her in- 
dividual reason for attending our program. 
I try to find out a little about their past 
educational experience. Once I have gotten 
a feel for where they may be academically, 

I decide which test to give them. We have 
different tools and tests that we use to as- 
sess students depending on their abilities. 

We use the TABE test when a student 
attended school for several years. For 
many students who come into our pro- 
gram the TABE test is very intimidating, 
not only the content but the test itself. If 
we see that a student is very nervous then 
we start them with our one page in-house 
assessment. The students see one page in- 
stead of many pages and the undertaking 
does not seem so overwhelming. 
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We then feel we are able to get a better 
understanding of the person's ability. The 
test has basically the same subjects as the 
TABE: whole numbers, computation and 
concepts, geometry and algebra. The main 
difference is it only has 25-40 questions, 
not 90 like the TABE. 

Our goal is to place students in appro- 
priate levels, but also to have them walk 
out of the assessment protocol with a posi- 
tive attitude about school. Many of our 
students have had negative prior experi- 
ences in school. They began to believe that 
they were incapable of doing math. We of- 
ten hear: 'I can't do that.' In order for these 
students to be successful in their educa- 
tional endeavors, we must have them over- 
come the belief that they cannot do it. 

Teachers also take part in the assess- 
ment process. Once the students are placed 
in the class the teachers assess them on a 
daily basis through class participation, 
classroom observation, formal testing, 
informal testing, informal testing projects 
feedback between the student and teacher, 
and in some cases working with math 
games on the computers. 

Twice a year teachers in the higher level 
classes use the TABE test to compare the 
scores the students had at the start of the 
class to that of the midterm and the final. 
The majority of the students are comfort- 
able with testing at this point and are anx- 
ious to see the progress they have made. 

Testing in the lower levels is not as 
structured. The teacher might opt to do a 
formal test at the end depending on the in- 
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dividual student. Many times the teacher 
looks at the student's portfolio and as- 
sesses using what they have collected over 
a period of time. If a student is to move on 
to a higher level we need scores docu- 
mented; therefore formal testing is done. 



After many years of actual testing and 
evaluating students we believe our new 
system works well. Enrollments high and 
retention is impressive. Very few students 
have dropped out in the last year. We hope 
that some of these assessment ideas will be 
helpful to your program. 



(chart follows) 
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Taunton Math Pre-Test 

Write the value of the underlined digit 

1. 612.480 2.324 

3. 15,687 4. 999 

Round to the place value named. 

5. 785 to the nearest ten 6. 15 r 642 to the nearest thousand 

ADD. 

7. 454 + 27 + 1,078 = 

8. 83,225 +7,453 + 54 = 

SUBTRACT. 

9. 865 ID. 756 11. 90,900 

- 360 - 638 - 78.221 

MULTIPLY. 12. 58 X 27= 13. 94 x 63 = 



DIVIDE. 14. 5 12,430 15. 23 ("2,254 

SOLVE THE FOLLOWING PROBLEMS. 

16. 2 + 5(3) = 17. 1 ♦ 8(3) = 



18. 5( 4+3) = 



19. 22 ♦ 3(8-5) * 



20. BCC has an intramural soccer league. A total of 440 students sign up for soccer. If 
each team has 22 players, how many teams will there be? 
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ROUND EACH NUMBER TO THE PLACE VALUE NAMED. 

21. 0.25 ( nearest tenth) 22. 9.376 ( nearest whole number) 

23. 0.2136 ( nearest thousandth) - 24. 8.607 ( nearest hundredth ) 

ADD OR SUBTRACT. 

25. 82.152 + .029 = 26. 87.31 - 29.8 = 

MUTIPLY OR DIVIDE. 

27. 36 X .25 = 28. 0.84/11 

SOLVE THE FOLLOWING PROBLEM. 

29. Mr. Amaral travels for three days. He spends $402.77, $398.89, and $228.24. How much 
did he spend? 



REDUCE EACH FRACTION TO LOWEST TERMS. 

30. 3 31. 24 

9 32 

CHANGE EACH MIXED NUMBER TO AN IMPROPER FRACTION. 

32. 3^ 33. I 75 

SOLVE THE FOLLOWING. 

34.2 X 4 = 35.5 - 2 = 

3 7 £3 



36. It takes Ellen of an hour to knit the back of a child’s sweater, 4 of an hour to 
knit the front, and 1 hours to assemble It. How long does it take Ellen to complete 
one Child’s sweater? 



37. Find the perimeter of the following figure. 38. SOLVE. 

S = P- 0.25P 



c~s . 

7.j? 



If S = $30, find P 

BEST COPY AVAILABLE 
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Learning and Change: 

A Phase Two North Carolina ESOL 
Framework Inq uiry Pro ject 



I n the spring of 1999 1 participated in Lit- 
eracy South's Phase Two of the North 
Carolina ESOL Curriculum Framework 
Inquiry Project. The purpose of this project 
was to discuss and analyze the draft form 
of the North Carolina ESOL Curriculum 
Framework, developed in a year long pro- 
cess by a group of 15 adult ESOL practitio- 
ners. 

Following our initial meeting, I focused 
my part of the inquiry project on one of the 
eight guiding principles developed by 
Phase One participants that asserted that 
language learning is a change process, both 
cognitive and affective. Further, that ESOL 
learners make non-language gains such as 
increased self-confidence and self-esteem. I 
specifically wanted to explore the non-lan- 
guage gains learners make in the ESOL 
classroom. I spent two months reading any 
literature I could get my hands on regard- 
ing the subject, talking to my colleagues 
about the changes they saw in their learn- 
ers, and conducting an inquiry with my 
ESOL learners. In this article I share the 
findings of my inquiry and what I see as 

Beth Brockman S ° me im P lications for ES0L practitioners. 




Durham, NC 



MY INQUIRY 

I decided to ask the ESOL learners in my 
intermediate level class at Wake Technical 
Community College what changes they 
have experienced while studying En glis h. 

I designed a survey entitled, "Are you 
changing?" based on a tool from East End 
Literacy (Hemmindinger, 1988). I taught 
this class on Monday and Wednesday eve- 
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nings, from 6:30 to 9:30. Students also at- 
tended class on Tuesday and Thursday, 
same time, but with a different teacher. 

I began class by telling the learners I 
was part of a research project and that I 
was interested in finding out the kinds of 
changes students had made in the process 
of learning English other than gains in lan- 
guage ability. We brainstormed a list of 
non-language gains students had made, 
which I wrote on the board. Then I gave 
out the assessment tool, "Are you chang- 
ing?" and briefly went through it to make 
sure the students understood all the vo- 
cabulary. The survey also included the re- 
quest, "If you can, draw a picture of how 
you looked when you first started to speak 
English and how you look now when you 
speak English." The students then com- 
pleted the survey individually. Finally, 
students got together with a partner and 
each person shared what she or he had 
written. 

The list of non-language outcomes the 
students brainstormed consisted of: 

• More confidence to go shopping 

• Ability to go out and make friends 

• Confidence to express feelings 

• Improved relationships— found love [!] 

• Come to class regularly, even every day 

• Travel more 

• Changed appearance, clothing 

• Better understanding of formal situa- 
tions 
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• Able to give help to family 

• Got driver's license. 

We discussed the changes as we 
brainstormed this list. Sometimes I asked 
clarifying questions. At other times I asked 
students to expound on what they had 
said, or to give an example. 

All the students agreed that they had 
changed. They also agreed that the amount 
of time they had spent in the U.S. and 
studying English influenced the changes 
they had made in their lives. They felt, 
generally, that they had made significant 
language gains after they had come to the 
U.S. and had studied in class. 

The answers on the survey varied. All 
the students listed language as. well as 
non-language gains they had made in the 
process of learning English. The categories 
of non-language outcomes the students' re- 
sponses fell into were: Confidence, Learn- 
ing Skills, Cultural Awareness, Knowledge 
of Social Institutions, Access and Entry 
into Further Study, and Support in the 
Learning Environment. 

IMPLICATIONS FOR ESOL EDUCATORS 

I was struck by three things in the sur- 
vey results. First, I noticed that students 
had difficulty separating non-language 
and language outcomes. Second, I was 
amazed at the level of excitement students 
exhibited in talking about how they had 
changed. Last, it was interesting to me that 
so many students were eager to draw a 
picture to show how they had changed. 

Of course it is understandable that stu- 
dents have difficulty separating language 
and non-language outcomes. It is difficult 
to talk about change without describing it 
holistically. The recommendations here for 
ESOL educators are twofold, I believe. 

O 
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First, non-language outcomes should be in- 
cluded in our objectives, right alongside 
our objectives for gains in language profi- 
ciency. Second, the methods used in our 
classroom should promote non-language, 
as well as language gains. 

Australia's Adult Migrant English Pro- 
gram (AMEP) shows its high regard for 
non-language outcomes in English lan- 
guage and literacy programs by incorpo- 
rating them within a Certificate in Spoken 
and Written English Competencies (Jack- 
son, 1994). "By doing this, full recognition 
is also given to the fact that non-language 
outcomes are amenable to... expression... 
within a competency-based training" (p. 
19). Non-language gains are not merely by- 
products; they are consciously addressed 
in the curriculum. AMEP recognizes that 
language and non-language outcomes are 
intertwined. 

An Australian study lists numerous 
achievements by incorporating non-lan- 
guage outcomes as an integral part of the 
development of language and literacy 
competencies (Jackson, 1994): 

• It makes these outcomes apparent 
within the framework of other educa- 
tional accomplishments. 

• It acknowledges the skills development 
in this area as part of an ongoing educa- 
tional process. 

• It encourages closer examination of the 
content of these outcomes. 

• It allows similar continuity of develop- 
ment in this area of skill as in others. 

• It fully appreciates the value of these 
outcomes to the proposed education 
both within and outside the context of 
the classroom. 

• It promotes the development of teaching 

7V 



ADVENTURES IN ASSESSMENT 



Volume 

12 * 62 




Teachers ofESOL 
students who want 
to portray a more 
authentic picture 
of their students 
should foster non- 
language outcomes 
in their students 
by creating assess- 
ment tools that 
give opportunities 
for students to talk 
about and reflect 
on the changes they 
are going through 
while studying 
English. 



practices specifically targeting these 
outcomes. 

This leads very well into the second rec- 
ommendation I have for ESOL educators: 
use methods in our classroom that pro- 
mote non-language, as well as language 
gains. I found that the Australian study 
seemed to place great emphasis on the 
methods used in an ESOL classroom. 
Specifically, the study mentioned that 
applying the principles of adult learning 
increased students' confidence and self- 
esteem (Jackson, 1994). 

Susan Imel lists the following adult 
learning principles to guide our methods 
in class: 

• Involve learners in planning and imple- 
menting learning activities; 

• Draw upon learners' experiences as a 
resource; 

• Establish a climate that encourages 
and supports learners and enhances 
self-esteem; 

• Encourage self-direction in learners; 

• Promote a spirit of collaboration in 
the classroom; and 

• Use small groups to encourage coopera- 
tion and promote teamwork (Rosen, 
1999). 

I would like to return to my surprise at 
the level of excitement students exhibited 
in talking about how they had changed 
and that so many students were eager to 
draw a picture to show how they had 
changed. I believe this has implications 
specifically in the area of assessment. If 
students are so eager to talk about and 
draw the changes they are making while 
learning English, that tells me this is some- 
thing worthy of being assessed! 
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I believe there is a lack of assessment 
tools that consider non-language out- 
comes. Certainly standardized measures 
fail to assess these gains. Thus, it is the re- 
sponsibility of educators, both instructors 
and administrators, to come up with tools 
that assess non-language outcomes. About 
88 percent of the teachers interviewed for 
the research in the Adult Migrant English 
program in Australia "thought that 
achievements in the non-language out- 
come area deserved formal recognition 
and should be recorded as part of an ad- 
equate student profile" (Jackson, 1994, p. 
8 ). 

Teachers of ESOL students who want to 
portray a more authentic picture of their 
students should foster non-language out- 
comes in their students by creating assess- 
ment tools that give opportunities for 
students to talk about and reflect on the 
changes they are going through while 
studying English. Teachers should encour- 
age students to write about their own ex- 
periences of change. For low-level literacy 
students, a teacher could use the Language 
Experience Approach (LEA). There are 
many variations of the LEA and it can eas- 
ily be adapted to most classroom situa- 
tions. 

For students with higher levels of lit- 
eracy, teachers can give them a variety of 
ways to share their stories through writing. 
Students can write in journals or dialogue 
journals, compose poetry, develop a stu- 
dent newsletter, and even publish a class 
newspaper or book. Students who publish 
their writings, either within or beyond the 
classroom, experience many benefits. They 
discover that the realities of their own lives 
are worth thinking about, getting down on 
paper, and sharing with other people 
(Peyton, 1993). These stories can be used to 
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help students think about the changes they 
are experiencing, and give them an avenue 
for expressing themselves. 

Seeing the students excited about ex- 
pressing themselves through drawing em- 
phasized to me the importance of 
acknowledging a variety of ways of know- 
ing and communicating (Schneider & 
Clarke, 1993). Thus, assessment should 
rely on an assortment of tools to reflect stu- 
dents' knowing and allow them different 
ways to express their outcomes. Certainly 
one assessment tool will not capture it all. 

CONCLUSION 

Participating in Phase Two of the ESOL 
Curriculum Framework Inquiry Project 
was a great learning experience for me. At 
the end of my project, I concluded that 
change is an important part of a healthy 
system. It is an essential and natural part 
of living, and plays a meaningful role in 
growth and evolution. Change and contin- 
ued creation signal new ways of maintain- 
ing order and structure. Thus, it is natural 
to expect ESOL learners in our classrooms 
to experience all kinds of changes, includ- 
ing those that fall into the realm of non- 
language outcomes. Instead of merely 
anticipating these changes, let's do what 
we can to encourage them by including 

M 



non-language outcomes in our objectives, 
addressing them in our classrooms, and 
designing assessment tools that capture 
these gains.. 
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Are you changing? 

Are you getting more confident? 

'A Are there things that you do now that you did not do before coming to English 
class? What are they? 

EXAMPLE: I go out by myself more often. 



★ How often did you come to English class when you first started studying 
English? Please explain. 



^ How often do you come to class now? Please explain. 



★ Do you feel less shy with your teacher now? Please explain. 
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★ Do you feel more comfortable participating in class? Please explain. 



★ When you first came to English class, did you speak English outside class? 
□ Yes □ No 

If yes, how often? Where? Describe your experiences. 



★ Do you speak English more in your every day life now than you did before you 
started studying English in class? Please explain. 



★ Are you more comfortable speaking English with people outside class? Please 
explain. If you can write an example, please do. 



more 
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if Are there any other changes you have made since you started studying English 
in class? What are they? 



if If you can, draw a picture of how you looked when you first started to speak 
English and how you look now when you speak English. 



Adapted from Hemmendinger, A. (1988). A tool kit: Self evaluation exercises for students and 
literacy workers. Ontario, Canada: East End Literacy. 
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